<y 

\s 

{N 

C3 
>  Cl 

k  -■  ^  ^ 

3 

\ 

■:’:p 

V 

>• 

3 

a- 

THE  APPLICATION  OF  A  THEORY 

>«'■  -i, 

,  -• — >*•«>«■ 

OF  MULTI-LEVEL  SYSTEMS 

ooi 

Si-'flC 

*cu;‘  C'O 

c:) 

TO  OPTIMIZATION  PROBLEMS 

V 


JERRY  L.  SANDERS 


FEBRUARY,  1963 


ASTI  A 
!’n:a?,nrirpnn  a 


'P' 

1 


SRC  24-A-63-I2 


I 

I 

1 

Jl 

I 


I 

I 


THE  APPLICATION  OF  A  THEORY 
OF  MULTI-LEVEL  SYSTEMS 
TO  OPTIMIZATION  PROBIEMS 


Jerry  L.  Sanders 


I 

I  February,  I963 

I 

I 

SRC  2U-AH53-12 


I 


FOREWORD 


The  Systems  Research  Center  is  a  research  and  graduate 
study  center  operating  in  direct  cooperation  with  all  departments 
and  divisions  of  Case  Institute  of  Technology.  The  center  brings 
together  faculty  and  students  in  a  coordinated  program  of  research 
and  education  in  the  important  techniques  of  systems  theory,  develop¬ 
ment,  and  application. 

Research  leading  to  this  report  was  carried  on  by  Ur.  Jerry 
L.  Sanders,  Graduate  Assistant,  under  the  direction  of  Dr.  Mihajlo 
D.  Mesarovic,  Associate  Professor  of  Engineering  at  Case  and 
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ABSTRACT 


An  attempt  is  made  to  simultaneously  develop  certain 
aspects  of  a  theory  of  multi-level  systems  and  decomposition 
techniques  for  multi -variable  optimization  problems. 

Two  level-three  goal  systems  are  explored  intensively 
with  particular  reference  to  the  method  of  '♦coordination''  employed 
by  the  second  level, unit.  •♦Direct"  and  "Indirect  Intervention** 
modes  are  explored.  The  problem  of  the  quantity  of  infornstion 
required  by  the  second  level  for  the  coordination  of  the  first 
level  systems  is  investigated.  "Direct"’  and  "Indirect  Models" 
are  investigated.  The  concept  of  the  "Indirect  Model"  leads  to 
the  development  of  a  Decomposition  principle  for  Non-Linear 
Programming.  The  concluding  chapter  presents  the  analysis  of  a 
numerical  example  coded  for  the  computer  which  demonstrates  the 
utility  of  certain  of  the  techniques  in  the  context  of  a  specific 
application. 
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1.  INTRODUCTION 


1.1  The  General  Approach 

In  this  thesis  we  will  carry  out  two  parallel  lines  of 
development.  We  will  derive  some  specialized  mathematical  techniques 
for  the  decomposition  of  large  ootimization  problems.  We  will,  in 
turn,  attempt  to  place  a  correspondence  between  these  mathematical 
results  and  the  processes  of  decentralized  control  such  as  one  might 
find  in  a  decentralized  organizational  structure.  We  will  also  use 
the  process  in  reverse,  e.g.  we  v/ill  consider  certain  qualitative  con¬ 
cepts  from  the  literature  on  human  organizations  to  suggest  avenues 
of  approach  for  the  development  of  -the  mathematical  aspects.  De¬ 
centralized  decision  making  systems  have  been  defined  by  A.  Whinston. 

**By  a  de  centra  la.  zed  decision  making  system  v;e  refer  to  the  following: 

Given  m  decisions  or  actions  to  be  made  and  n  decision  makers  (1  <  n  -^'m), 
each  decision  maker  is  assigned  a  subset  of  the  m  decisions.  For 
the  over-all  system  there  is  given  a  criterion  function  and  a  space 
of  possible  choices  involving  the  m  decisions.  Each  decision  maker 
is  assigned  a  space  of  possible  choices  and  a  criterion  function  in¬ 
volving  at  least  the  decision  variables  he  can  partially  or  totally 
control." 

We  hone  by  adopting  this  dual  point  of  view  to  simultaneously 
develop  a  framevcork  for  the  synthesis  of  "organization-like"  structures 
and  at  the  same  time  use  the  mathematical  interpretation  of  these 
"organizational  structures"  to  develop  efficient  computational  algorithms 
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for  large-scale  optimization  problems.  Our  results  v/ill  have  applica- 

(1)^ 

tion  to  engineering  control  problems'  '  as  well  as  certain 

optimization  problems  which  arise  in  the  production  theory  of  the 
("13^^ 

firm.'  We  mention  in  passing  the  similarity  of  the  problems 

encountered  in  these  two  areas  to  problems  which  arise  in  macro- 
economic  theory.  Extensive  investigation  of  control  problems  in 
economic  theory  is  currently  underway  at  Case.^^^^  We  will  not  ex¬ 
plore  this  latter  area  in  any  detail. 

Large-scale  optimization  problems  have  been  a  constant 
source  of  trouble  to  systems  engineering  and  operations  research  since 
their  inception.  Roughly  speaking  an  optimization  problem  will  be 
considered  "large”  when  the  computational  requirements  which  must  be 
satisfied  in  order  to  find  the  optimal  valiie  of  the  manipulated 
variables  exceeds  the  capacity  of  current  computing  machinery  or  -when 
the  quality  of  the  performance  of  the  system  decays  significantly 
in  the  time  required  to  compute  a  new  control  solution. 

To  our  knowledge  no  previous  investigation  has  been  under¬ 
taken  7/ith  the  same  point  of  view  as  adopted  here  although,  of  co-crse, 
we  draw  on  the  results  of  many  investigators  working  in  similar  areas. 
Perhaps  a  combination  of  two  papers  have  come  as  close,  in  spirit  at 
least,  to  our  area  of  Interest  as  any.  These  are  l^inston  (ibid.)  and 


lems 


See  these  references  for  discussion  of  optimization  prob¬ 
in  these  areas. 


3 


Dantzig  and  TTolfe. We  quote  from  the  abstract  of  the  latter, 

«... Besides  holding  promise  for  the  efficient  computation  of  large- 
scale  systems,  the  principle  yields  a  certain  rationale  for  the 
1  decentralized  decision  process'  in  ,,  .  theory  of  the  firm....** 

The  first  paper  mentioned  above  is  essentially  a  detailed  investi¬ 
gation,  both  descriptive  and  analytic,  of  the  consequences  of  this 
statement. 

1.2  Types  of  Problems  Considered 

We  are  interested  here  in  "large**  multivariable  optimization 
problems  of  the  type  v/hich  often  occur  in  the  control  of  complex 
systems.  Subsequently  the  reader  Tri.ll  often  encounter  the  term 
"control  problem**.  This  term  should  be  considered  to  be  identical 
in  meaning  to  the  phrase  *».. .optimization  problem  which  arises  in  the 
optimal  control  of  a  complex  system...."  An  exact  definition  of  what 
is  meant  by  "control  problem"  will  be  given  in  Chapter  2. 

We  attempt  an  '’organizational**  approach  to  complex  control 
problems,  i.e,  we  attempt  to  break  up  the  over-all  control  problem 
into  a  number  of  smaller  problems,  each  of  which  is  to  be  solved  by 
a  real  or  fictitious  "first  level"  control  unit.  In  addition  we 
synthesize  one  or  more  "second  level"  control  units  udiose  fuinction  is 
to  coordinate  two  or  more  first  level  controllers.  By  proceeding  in 
this  way  we  hope  to  a.chieve  the  following  economiest  if  the  process 
is  a  real  one,  i.e.  if  the  imagined  organizational  structure  can  be 
realized,  then  we  will  enjoy  the  benefits  of  parallel  operation  in 
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that  several  parts  of  the  over-all  problem  will  be  processed  simultan¬ 
eously,  Xi“  the  process  is  imaginary,  l.e,  is  simply  a  computational 
device,  then  we  have  traded  the  task  of  solving  a  large  problem  for 
that  of  solving  a  number  of  smaller  ones.  In  either  case  this  pro¬ 
cedure  can  lead  to  significant  computational  savings  as  we  will  see. 

It  will  not  always  be  possible  in  the  models  that  we  examine 
to  achieve  the  same  quality  of  performnce  in  the  "organizational” 
approach  as  we  might  have  obtained  had  we  been  able  to  solve  the 
original  problem.  In  the  cases  vdiere  we  do  not  achieve  the  true 
optimum  we  will  attempt  to  offer  a  means  of  computing  the  difference 
in  performance  between  our  method  and  the  "true  optimtim"  performance. 

The  consideration  of  the  best  method  for  breaking  un  the 
original  control  problem  into  its  respective  parts,  i,e,  "optimal 
reticulation",  is  not  one  of  the  problems  considered  here.  This 
problem  is  intimately  connected  with  the  concept  of  "interaction". 

No  generally  accepted  definition  of  this  term  as  yet  exists.  Exten¬ 
sive  work  on  this  and  related  problems  is  currently  underway  at  the 
Case  Systems  Research  Center. qualitatively,  at  least,  we  would 
like  to  break  or  "reticulate"  the  system  equations  at  those  points 
fdiere  the  interaction  or  coupling  is  weakest.  In  the  problems  we 
consider  we  will  either  assume  that  the  reticulation  is  given  a 
priori  or  we  will  perform  the  reticulation  more  or  less  arbitrarily. 


without  regard  to  the  strength  of  interaction 
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2.  DEFINITIONS  AMD  PRELIMINARIES 


Introductiui 

In  thia  chapter  we  will  state  a  rigorous  definition  of  what 
we  mean  by  *»Control  Problem**,  indicating  what  information  is  necessary 
in  order  for  a  control  problem  to  exist.  **Multi-level  Control**  is 
defined  and  the  synthesis  procedure  which  will  be  investigated  in 
subsequent  chapters  is  discussed.  The  level  of  rigor  that  the  reader 
can  expect  and  the  exploratory  nature  of  the  entire  investigation 
are  explained. 


2.1  Definition  of  the  Control  Problem 

Since  we  are  considering  the  applicability  of  our  results 
to  three  rather  dissimilar  fields,  we  will  require  a  somewhat  differ¬ 
ent  approach  to  control  problems  than  has  customarily  been  taken  in 
the  literature. 


For  our  purpose,  in  order  for  a  control  problem  to  exist, 
we  require  three  separate  statements. 

1.  Specification  of  the  utility  functional.  Here  we 
assume  that  there  is  specified,  a  priori,  a  func¬ 
tional  g( . ,  .,  ..  .)  defined  over  the  Cartesian 
nroduct  space  of  the  variables  m,  ...  m  ,  i.e.  over 

X  M2  X  •••  ^ 

2,  Specification  of  Restrictions  on  the  Variables  m^ . 

Ife  assurae_that  there  is  specified  In  advance  a  vector 
function  ^  of  the  variables  m^  ...  m  which  specify 
the  restrictions  that  the  **systom**  places  on  the 
problem.  Whitten  out  these  conditions  aret 

(m^  ...  ra^)  <0 

« 

4 .  (m^  . . .  m  )  <  0 
'  1  n'  — 
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Exanple  1.  For  certain  engineering  problems  and 
for  many  economic  models  these  restrictions  take 
the  form  "^  =  0  where  ^  =  y  -  Am  -  C .  y  is  a 
n  X  1  vector  of  outputs,  A  is  a  n  x  n  matrix,  m 
is  a  n  X  1  vector  of  manipulated  variables  and 
Cis  a  n  X  1  vector  of  disturbances# 

Example  2 .  In  the  production  theory  of  the  firm  we 
often  encounter  the  following  type  of  restrictions: 

Am  -  F  <-0  where  a  typical  element  of  this  vector 
matrix  inequality  might  for  example  express  the 
fact  that  the  total  operating  tin*  for  any  one 
facility  is  less  than  or  at  most  equal  to  2li  hours 
in  any  one  day. 

3«  Specification  of  the  Environment.  In  general  we  will 
assijme  that  the  variables  m^  ...  m^^  are  controllable  by 
the  system  or  the  system  manager  am  that  the  variables 
. . .  m  are  not  under  the  direct  control  of 
tne  system  mafiager  although  in  some  cases  these 
variables  may  be  observable.  The  specification  of 
the  environment  will  consist  of  the  specification 
of  a  probability  density  function  h(m.  ...  m  ) 

which  exoresses  the  joint  probability  of  occurrence 
of  specific  numerical  values  of  the  external  variables. 
The  density  h(raj^  ...  m  )  may  take  several  forms. 

It  may  be  a  true  probability  density  function  or  a  sub¬ 
jective  probability  statement  or  a  statement  of  the 
exact  values  ^  m  will  take  on  (in  the 

deterministic  case;  or  the  set  (k  +  1,  . . .  n J  may 
be  empty.  We  will  often  substitute  the  vector  e 
for  the  variables  ...  m  in  the  chapters 

that  follow.  ^ 


Now  given  these  three  statements  we  can  formulate  the  complete 
control  problem  as  follows: 

maximize  or  minimize  E  g(m^  ...  m^,  m^  ...  m^)  (2.1) 

with  resnect  to  m, ,  ...  ra, 

1’  k 

subject  to  the  restriction^  (m^  ...  m^)  -<0 

■vrtiere  E  is  the  operation  of  mathematical  expectation 

over  the  variables  m,  ^  ...  m  , 
k  +  1  n 
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Example  1 

—  —  T  —  — 

minimize  E  (Am  +  £)  B  (Am  +  £) 

over  in  ^ 

■where  £ia  distributed  Tfith  p.d.f  ^(£  ), 

or  stated  in  the  customary  wayi 

T 

minimise  E  y  By  subject  to  the  constraint 
over  m 

y  -  Am  -  f  =  0 
Examnle  2 

N 

_  min  _  4  ^  ® 

m^>  . . ,  m^  i— 1 

subject  to  y^  =  5"  (a  cons'tant  vector) 

*  1  + 1-*  + 1 
■where  ^  is  distributed  ■with  p.d.f  )jj  j  +  1^* 

All  the  subscripts  in  this  example  refer  to  a  time  index  and  not  to 

a  vector  component. 

Example  3  „ 

maximize 

P 

subject  to  <3.  m  -  b .  <■  0  i  =  1, 

j-1  ^  ^ 

■where  ,  and  b^^  are  all  fixed  constants. 

The  reader  ■will  of  course  recognize  this  as  the  classic  linear  pro¬ 
gramming  problem. 

2.2  Optima,!  vs.  Sub-Optimal  Control 

The  solution  to  the  problem  posed  in  2.1  will  be  referred 
to  as  the  Optimal  Control  Solution.  By  solution  here  we  mean  the 
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optimi7,lng  values  of  ,,, 
follow  we  will  occasionally  consider  ways  of  generating  the  controllable 
variables  -viiiich  do  not  yield  the  same  values  as  those  generated  in, 
the  solution  to  problem  2.1.  The  solutions  7ri.ll  be  referred  to  as 
Sub-Optimal  Control  Solutions..— 

The  work  "optimal"  here  is  \mfortunately  a  "loaded"  word. 

The  solution  to  2.1  is  the  "best  that  one  can  do"  in  the  context  stated j 
however,  it  neglects  one  critical  economic  factor — ^the  cost  of  com¬ 
putation.  One  might  well  abk-  "What  is  ' optimal"  about  a  solution  that 
7 

will  take  10  years  to  calculate  on  present  computing  machinery?"  or, 
"What  is  'optimal'  about  a  solution  for  a  chemical  processing  plant 
idiose  expected  net  profit  for  the  year  is  ^10,000  when  the  computing 
machinery  required  for  the  comoutation  of  the  control  solution  costs 
on  the  order  of  J^10°?"  Contrary  to  being  unusual,  examples  such  as 
those  just  mentioned  are  quite  prevalent  in  practice. 

Very  little  work  has  been  done  on  over-all  methods  of  opti¬ 
mization,  i.e.  general  economic  optimization,  in  control  processes. 
However,  the  cost  of  comoutation  considerations  indicates  that  we  need 
to  examine  certain  of  our  prejudices  concerning  general -distaste  for 
so-called  "sub-optimal"  so.Lutions,  because  it  is  easily  possible  that 
these  "sub-optimal"  procedures  are  actually  economically  preferred  to 
the  "optimal"  solution. 


X  *  Q  •  M.. 


In  -the  chapters  that 
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2.3  Single  and  Multi-Level  Control 

For  the  purposes  of  illustration  we  will  consider  some  block 
diagrams  of  some  of  the  control  processes  that  will  appear  later. 
Nearly  all  of  the  control  processes  which  occur  in  engineering  and 
many  of  those  which  occor  in  operations  research  can  be  represented 
as  follows 1 
Fig.  2.1 


C  is  the  '•causal  unit**  or  •♦system**  or  "plant**,  G  is  the  "goal  seeking 
unit"  or  •* controller".  This  unit  is  responsible  for  maintaining  a 
model  of  the  process  inherent  in  C  and  is  assumed  to  have  a  utility- 
function.  G  also  has  a  model  of  the  en-vrironment  which  yields  the  in¬ 
puts.  The  utility  function  is  a  function  of  the  inputs  i  the  outputs 
^and  the  controllable  variables  m^.  The  variables  m^  indicate 
possible  measurements  performed  by  G  on  C.  G  is  assumed  to  choose 
those  manipulated  variables  m^  which  maximize  its  utility  function 
subject  to  the  models  it  has  of  the  environment  and  the  process. 

We  will  refer  to  systems  such  as  Fig.  2,1  as  iZ-y  systems .  This 
notation  refers  to  the  fact  that  the  structure  is  "single  level"  and 
•♦single  goal'*.  By  "single  level**  we  mean  roughly  that  no  managing 
or  coordinating  controllers  are  present.  By  "single  goal"  we  mean 
that  only  one  utility  function  is  involved  in  the  entire  system. 
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Almost  the  entirety  of  this  thesis  will  be  devoted  to  the 
study  of  the  analysis  and  synthesis,  of  Ijf  ~  3^ systems.  Their  block 
diagrams  will  look  either  like  that  in  Fig«  2.2  or  that  in  Fig*  2.3. 


is,  of  course,  the  ^coordinating  unit"  or  "managerial  ■unit"  or 
"second  level  controller".  is  entrusted  -vd-th  the  utility  function 
of  the  entire  system  and  with  various  models  of  the  "behavior"  of  G^, 
C^j  ^2*  ^2  environment. 

The  arrows  1^  and  represent  the  flow  of  information  up 
from  the  "first  level"  systems  and  1^  and  k^  represent  the  flow  of 
control  do'wn  from  G^*  The  system  represented  by  Fig.  2.2  is  dis¬ 
tinguished  from  2.3  by  the  fact  that  G^  is  employing  a  method  of 
"coordination"  which  we  •will  call  "indirect  intervention"  because 
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we  note  that  G^'s  attempts  to  steer  the  i^ole  system  are  made  via 
control-action  aimed  at  the  goal-sooking  units  G-j_  a^nd  Gg* 

2.3  we  call  this  coordination  mode"  '♦direct  intervention’!  because 
the  second  level's  control  action  is  directed  toward  the  causal  systems 
and  Cj* 

Example  1  windirect  Intervention'* 

Consider  a  decentralized  firm  with  two  operating  divisions 
and  a  central  planning  unit  where  the  central  planning  unit  sets  the 
internal  or  accounting  prices  of  certain  commodities  such  as  staff  ser¬ 
vices,  etc.  In  this  example  the  central  planning  unit  plays  the  role 
of  Gj  and  the  two  operating  divisions  are  represented  by  (G^,  C^)  and 
(G2J  02)*  The  control  actions  1^,  and  are  the  setting  of  the  price 
of  staff  services  for  the  respective  divisions.  The  method  of  co¬ 
ordination  here  is  '♦indirect'*  because  the  central  planning  unit,  by 
setting  the  prices,  is  essentially  manipulating  parameters  in  the 
utility  functions  of  G^  and  G2* 

Example  2  "Direct  Intervention" 

Consider  an  international  planning  agency  which  is  responsible 
for  the  economic  growth  (suitably  defined)  of  two  underdeveloped 
countries.  The  separate  governments  are  represented  by  G^^  and 
Assume  that  the  planning  agency  decides  to  give  each  country  a  large 
sum  of  money  to  be  immediately  injected  into  the  respective  economics 
(Cl,  Cg)  in  the  form  of  construction  spending,  etc.  This  method  of 
"control"  would  be  "direct  intervention"  because  direct  incuts  are  in¬ 
jected  into  the  causal  systems — in  this  case,  the  respective  economies. 
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Obviously,  these  distinctions  are  somev/hat  arbitrary  and 
may  or  may  not  be  identifiable  in  any  real  system.  However,  we  will 
find  these  distinctions  useful  in  the  chapters  to  follow.  Roughly, 
we  will  characterize  a  goal-seeking  unit  G  as  being  "second  level" 
if  its  primary  function  is  the  "coordination"  of  two  or  more  first 
level  units. 

By  "coordination"  we  mean  the  process  of  the  transmission 
of  information  and  control  signals  from  the  second  level  unit  to  the 
first  level  units  for  the  purpose  of  improving  the  over-all  performance 
of  the  system  as  measured  by  the  performance  functional  of  the  second 
level  unit. 

One  further  component  of  the  over-all  problem  from  the  point 
of  view  of  is  the  type  of  model  that  this  unit  has  of  the  structure 
"below"  it.  We  will  classify  the  model  types  as  either  "direct"  or, 
"indirect".  G^  is  said  to  have  a  "direct"  model  of  the  system  below 
it  if  it  requires  complete  knovrledge  of  the  structure  of  (G^^,  C^) 

(G2»  C2)  s^s  well  as  the  system's  environment.  is  said  to  have  an 
"indirect"  model  of  the  system  below  it  if  it  requires  only  the 
knowledge  of  the  system' s  environment  and  the  utility  functions  of 
Gj  and  G2  or  certain  parameters  of  these  utility  functions. 

2.U  Remarks  concerning  Rigor  and  Intent 

The  models  which  appear  in  the  following  chapters  are  in 
no  vfay  to  be  construed  to  be  final  answers  concerning  the  questions 
posed  in  this  thesis.  The  intent  here  is  purely  exploratory.  The 
amount  of  room  left  for  investigation  is  nearly  limitless  in  this 
area. 


I 
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In  keeping  with  the  exploratory  nature  of  this  investiga¬ 
tion  we  will  often  sacrifice  mathemaatical  rigor  to  the  gods  of 
expediency.  The  analytically  inclined  reader  may  be  somewhat  appalled 
by  the  extreme  assumptions  made  concerning  the  existence  of  inverses, 
derivatives,  etc. 


Summary 

In  this  section  we  will  summarize  in  outline  form  the  syn¬ 
thesis  procedui‘e  that  we  will  carry  out  several  times  in  the  succeed¬ 
ing  chapters.  It  should  be  emphasized  that  the  steps  outlined  below 
do  not  occur  in  any  particular  sequence  but  rather  must  more  or  less 
be  carried  out  simultaneously. 


Given  the  problem i 


extremize  E  g  (m,  ...  m  ) 

.  'in 

subject  to  the  constraints 
^(in^  ...  m^)  <-0 

and  an  environment  prob.  density  function  +  1  ***  "'n^ 

1.  Decide  on  a  division  of  control  labor  between  the 
two  first  level  systems.  (Reticulate) 

2.  Assign  objective  functionals  (utility  functions) 
and  constraints  to  the  first  level  systems. 

3.  Assign  an  environment  prob.  density  function  to  each 
first  level  unit. 


Ut  Decide  on  the  model  type  to  be  given  second  level 
unit. 

a)  Direct  Model, 

b)  Indirect  Model 


Decide  on  an  intervention  method  for  the  second 
level  unit. 

a)  Direct  Intervention 

b)  Indirect  Intervention 

Investigate  the  quality  of  performance  of  the 
resultant  system. 


3.  DIRECT  INTERVENTION  —  STATIC  CASE 


Introduction 

In  this  chapter  a  class  of  linear,  static  control  problems 
are  investigated.  The  synthesis  of  a  2£  -  controller  is  investi¬ 
gated  from  the  noint  of  view  of  "Direct  Intervention".  The  second 
level  unit  is  assumed  to  have  a  "Direct"  model  of  the  subsystems 
below  it. 

3«1  Model  1  Deterministic  Case 

The  Durpose  of  this  chanter  is  to  explore  several  possible 
models  of  control  situations  where  the  systems  are  said  to  be  "static", 
i.e.  either  the  control  action  is  applied  only  once  or  if  the  control 
action  is  applied  nany  times  it  is  assumed  that  the  effects  of  the 
applications  are  statistically  independent  of  each  other.  All  of 
the  systems  in  this  chapter  and  the  rest  of  the  thesis  as  well  are 
assumed  to  be  multi-variable,  i.e.  several  state  variables  and  several 
manipulated  variables.  The  system  equations  are  assumed  to  be 
"linear"  and  constant  over  time.  In  mathematical  terms  this  means 
that  the  system  acts  as  a  linear  operator  which  maps  elements  (vectors) 
of  the  "input  space"  into  elements  of  the  "output  space".  Our 
purpose  in  investigating  the  use  of  2^-  3^  controllers  for  linear 
systems  control  is  not  to  solve  any  new  problems  in  the  area  of  the 
analysis  of  linear  systems  with  a  small  number  of  state  variable s> 
since  these  problems  have  been  attacked  for  many  years  by  many 
authors  with  great  success.  Our  purpose  is  to  "cut  our  teeth"  on 
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a  class  of  problems  that  are  analytically  tractable  in  the  hope 
that  we  will  discover  methods  of  approach  idiich  will  suggest  effi¬ 
cient  ways  of  proceeding  in  very  complex  control  problems,  e«g« 
large  non-linear  problems  or  even  linear  problems  with  many  state 
variables  (e.g.  more  than  20). 

Our  general  anproach  in  these  problems  will  bo  as  follows i 
we  will  take  the  original  multi -variable  control  problem  and  attempt 
to  break  it  into  two  or  more  smaller  parts.  From  these  smaller 
parts  we  will  construct,  generally,  two  sub-optimal  control  problems, 
each  of  irtiich  neglects  the  possible  interactions  with  the  other 
problem.  At  this  point  the  second  level  of  control  will  be  intro¬ 
duced.  It  is  the  pumose  of  the  second  level  control  unit  to  intro¬ 
duce  into  the  system  at  an  aporopriate  point  a  signal  which 
compensates  for  the  neglected  interaction.  This  chapter  ttempts 
to  investigate  this  method  of  control  in  the  static  case  and  to 
compare  its  performance  with  possible  alternatives.  The  first  of 
these  alternatives  is  the  case  where  the  interaction  between  the 
two  original  "First  Level*  controllers  is  completely  neglected.  The 
second  alternative,  which  is,  in  large  complex  problems,  never 
available  to  us,  is  that  of  the  true  optimal  solution. 

For  simplicity  of  exposition  we  will  confine  ourselves  to 
operators  defined  on  finite  dimensional  vector  spaces.  A  theorem 
from  functional  analysis  tells  us  that  we  can  treat  the  finite 
dimensional  operators  as  finitie  matrices.  (See  Appendix  A). 
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Model  1  y  =  Am  +  c  ,  |a|  0 


(3<>1) 


composed  as  follows 


=  A 


iriioro  Aj^j^  is  a  k  X  k  submaitrix 


y  is  an  n-oomponent  vector Cy^^,  o..  »  T  is  used  to  denote  the 

transpose  of  a  vector  or  matrix.  A  is  assumed  to  be  an  n  x  n  matrix 

[^11  ^12  i 

4l  ^22! 

(k  does  not  have  the  same  meaning  hers  as  in  eq.  2,1).  Aj2 
k  X  (n  -  k),  is  (n  -  k)  x  k  and  A22  is  (n  k)  x  (n  -  k), 
m  and  €  are  of  the  same  dimensionality  as'y.  We  can  view  the  y 
variables  as  outputs  of  the  process,  lii  as  the  inputs  and  the  €  as 
distiirbances  arising  from  the  environment.  In  this  model  the  £  are 
assumed  to  be  deterministic  and  known.  The  vectors  yi  ni  and  ^  may 
either  be  considered  as  being  composed  of  numbers  or  as  being  composed 


of  time  functions,  e.g,  y  = 


yi(t) 


y;(t) 


.  The  important  difference 


between  the  models  of  this  chapter  and  those  of  chapter  5  is  that 

in  thivs  chapter  y^j^Ct  +  a)  is  independent  of  yj(t)  for  all  i,  j,  t  and 

a,  ife  assume  that  a  criterion  of  over-all  system  performance  is  given. 

It  is  also  assumed  that  the  second  level  unit  is  entrusted  with  the 

maximization  of  this  utility.  In  this  model  the  goal  of  the  system 

operator  is  assumed  to  be  to  minimize  Ty^  B  y7,  y"^  B  y  can  be  view- 

w.r.t,  m  3 

ed  as  the  cost  of  deviation  from  a  fixed  operating  point  which  in 
this  case  is  0.  The  operation  of  the  system  around  any  predescribed 
operating  point  can  be  accommodated  by  simply  changing  the  origin  of 
coordinates  in  the  space  in  idiich  y  is  measured.  In  other  words, 
the  job  of  the  system  operator  in  this  case  is  to  keep  the  ^stem  as 
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close  to  the  fixed  operating  point  as  possiblco  Certain  disturbances 
€  impinge  on  the  system  from  the  environment  but  the  operator  can 
change  the  vector  In  in  an  attempt  to  offset  these  disturbances.  The 

cost  of  deviating  from  the  preset  optimum  conditions  is  rou^ly 

— T  — 

proportional  to  the  square  of  the  dsriation  since  y  By  is  a 


quadratic  form.  B  is  composed  as  follows  B  = 


Bii  0 

0  B 


0  " 
22_ 


where  B-|  ^ 


is  k  X  k  and  (n  -  k)  x  (n  -  k).  (B^^^  is  assumed  to  be 

positive  definite).  At  this  point  we  introduce  some  notation  which 


we 


will  use  throu^outo  We  let  g  =  y"^  By  and  g*  =  min  ^y^  B 
g  then  is  the  actual  performance  and  g-K-  is  the  optimal  performance. 
We  will  now  assume  that  the  problem  of  determining  the  optimal  m 
is  too  large  a  problem  to  be  solved  by  current  optimization  techniques. 
We  will  therefore  subdivide  the  problem  as  previously  described.  We 
let  the  subvector  ...  yj^J  Similarly  |yj^  +  1  ^nj  ~ 

y  f  •••  =  m  ,  |mj^  ^  1  =  m  >  ...  ~  6  * 


and 


T  _  -2 

e 


We  now  turn  the  responsibility  for  y^  and  m^  over  to  a 

2  —2 

controller  called  "First  Level  Systemf^"  and  y  and  m  over  to 
"First  Level  System  System  1  is  set  the  following  task* 


=*u 


-r 


®ii  1^ 


-1 


subject  to 


(3.2) 


-1  -1 
m  +  g 


The  extra  subscript  introduced  here  is  to  denote,  for  example,  that 
is  System  1»  s  "model"  of  the  vector  y^.  The  reader  will  note 
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th>t  this  model  is  actually  incorrect  from  the  point  of  view  of  the 
original  problem  posed.  System  1  is  neglecting  the  element 
which  is  a  cross-coupling  term  coupling  y^  to  ,  System  1  neglects 
this  term  on  purpose  in  order  to  separate  its  problem  from  System  2's. 


System  2's  problem  is  to 


minimize 

subject  to 


®22  2^ 

-2  .  -2^-2 

2^  *22  ®  ^ 


(3.3) 


The  solution  to  problems  (3.2)  »nd  (3*3)  can  easily  be  derived  (see 
Appendix  A).  Equating  the  vector  derivative  of  the  quadratic  forms 
in  (3«2)  and  (3*3)  to  zero  we  have 

*22  ®22  ^*22  ^  ♦  ^)  =  0  C3.U) 

and  A.J  =  0  (3.?) 

Tfe  assume  (as  always,  for  the  sake  of  simolicity)  that 

T 

and  A^^  are  non-singular  and  that  A^^  A^^^  is  positive 

definite.  Therefore,  we  have 


.-1-1 

-  -All  ■€ 


and 


-2 

€  . 


We  note  that  the €'3  are  assumed  to  be  known  to  all  the 
systems  involved  in  the  control  of  the  process.  If  these  vectors 
m  were  allowed  to  be  implemented  as  they  are  the  actual  outputs  of 
the  process  would  be 


-J. 

-1* 

-2# 

-1 

..-1 

-2 

y 

“  *11 

m 

*12 

m  + 

G  ^12 

*22 

e 

-2 

-1# 

2« 

-2 

.-1 

j. 

y 

=  *21 

m 

+  A22 

m  + 

€  =  ^2L 

*11 

£ 
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A\t  this  point  we  will  introduce  the  second  level  controller. 
System  #3.  System  3  has  the  responsibility  for  the  control  of  the 
entire  process  and  as  such  it  will  be  concerned  with  the  complete 
model  of  the  system  and  with  the  system  performance  functional. 
Specifically  its  task  will  bo  to  insert  appropriate  inputs  into  the 
system  to  compensate  for  the  interaction  neglected  ty  the  first  level 
units.  To  be  more  precise,  we  will  assume  that  System  3  has  two 
optimization  problems. 


problem  1 


sxijject  to 
-2 


minimize 
-1 


-1" 


Bii  f 


_1 

-2 


(3.7) 


“  ^11  ^  ^12  ^ 


-2 

+  ^ 


where  is  a  k-component  vector  manipulated  by  the 
second  level  unit. 


o-t  O' 

problem  2  minimize  ^y  B22  (3«8) 

subject  to  ^  m^  +  A22  *  Agj  ^  ^ 

is  an  (n  -  k)  component  vector  manipulated  by  the 
second  level  unit. 


We  have  already  assumed  that  System  3  has  the  knowledge  of 
the  complete  structure  of  the  system  itself  as  well  as  the  optimal 
control  vectors  from  the  first  level  systems,  i.e,  m  and  in  . 
The  first  level  units  are  required  only  to  have  knowledge  of  their 
own  subsystems  and  their  part  of  the  system  goal  in  addition  to  the 


relevant  disturbance  sub-vector 
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-1 

-1 

-2 

.  -2 

NOW  ^ 

-  -A^2  A22 

6  + 

-2! 

-1 

-1 

.  -1 

3^ 

~  “^21  ^11 

e  * 

^21  ‘V 

The  solution  to  System  3*s  two  problems  is  now  obvious.  It  should 

-1  -2  -i*  -1  _2 

set  =  A22  ^  *1,  “  e  •  Therefore  ^y  =  ^y  =  0  and 

the  value  of  the  performance  functional  for  the  entire  system,  i.e, 

^  B  y  takes  on  the  value  0« 

Let  us  examine  briefly  the  imolications  of  the  results  of 

this  rather  elementary  example.  First  notice  that  the  system  per- 
— T  — 

formance  functional  y  By  assumes  its  absolute  minimum.  This 
means  that  no  other  control  algorithm  will  produce  better  result; 
for  this  system.  This  is  in  fact  then  optimal  control.  We  have 
taken  the  original  problem  apart  and  recombined  it  along  organizational 
lines  in  such  a  way  as  to  get  the  best  possible  performance,  A 
reasonable  question  at  this  point  is  *Tfhy?»  The  reason  here  and  for 
all  subsequent  models  is  computational.  In  nearly  all  real  multi- 
variable  control  problems  the  problem  of  computing  a  sequence  of 
optiimal  control  variables  meets  with  many  computational  difficulties. 
Assume  that  the  matrix  A  is  of  even  dimension.  If  we  had  tried  to 
solve  the  original  optimization  problem  directly  we  would  have  re¬ 
quired  the  inverse  of  the  n  dimension  A  matrix.  In  our  problem 
we  required  only  the  inverse  (k  =  of  matrices  of  size  The 
time  for  the  computation  of  the  inverse  of  a  matrix  on  a  digital  com¬ 
puter  goes  up  roughly  as  the  cube  of  the  matrix  size.  Here  and 
particularly  in  the  dynamic  case,  as  we  will  see  later,  this  kind  of 
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difference  c»n  mean  the  difference  between  being  able  to  solve  the 
problem  and  being  forced  to  neglect  certain  potentially  important 
aspects  of  the  performance. 

The  reader  can  undoubtedly  level  several  objections  at  this 
model.  The  first  might  be  that  this  might  not  be  a  feasible  approach, 
i.e.  to  intervene  in  the  performance  of  the  system  in  this  way,  and 
this  we  would  readily  admit.  Secondly,  one  mi^t  argue  that  we  are 
not  considering  the  cost  of  the  manipulated  variables.  This  will 
be  considered  in  subsequent  models.  Thirdly,  one  might  say  that 
linear  models  of  this  kind  are  so  naive  in  structure  and  formalism 
that  they  can  never  represent  the  control  of  a  real  process.  This 
allegation  has  a  certain  amount  of  truth  to  it  and  may  recur  in 
several  of  the  subsequent  models .  We  will  attempt  to  answer  it 
here  and  offer  no  further  apologies  later.  First  of  all,  it  is  not 
strictly  correct,  since  models  of  similar  type  have  been  employed 
in  both  engineering  and  economics  for  many  years  with  varying  degrees 
of  success.  The  majority  of  models  encountered  in  operations  research, 
however,  are  extremely  non-linear.  Unfortunately,  analytical  results 
are  almost  completely  lacking  for  the  control  solutions  to  non-linear 
^sterns.  Therefore,  whether  we  like  linear  models  or  not  we  are 
forced  to  them  in  order  to  try  our  approach  on  a  fairly  broad  class 
of  system  models.  The  digital  computer  offers  us  very  little  succor 
hero,  because  it  yields  only  one  solution  at  a  time  and  not  a  class 
of  soliitions  as  we  require. 
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Th«  interesting  feature  of  this  model  -was  that  ire  were 
able  to  achieve  the  true  optimum  with  our  method  of  control  system 
synthesis 0  As  we  will  see  subsequently,  this  is  not  always  possible 
with  this  method. 


3,2  Model  2 


Probabilistic  Case 


This  model  is  to  be  formulated  in  exactly  the  same  way  as 
Model  1,  except  that  the  disturbance  vector  t  is  assiimed  to  be  a 
vector  of  random  variables.  The  first  moments  of  all  the  components 


are  assumed  known  and  finite,  i.e. 


Ek. 


All  the  second 


order  moments  of  c  are  assumed  finite.  Now  we  can  no  longer  use 

— T  — 

f\motionals  of  the  type  y  B  y  as  an  optimization  criterion  be“ 
cause  these  functionals  themselves  become  random  quantitiesj  however, 
we  can  use  their  expectation  as  an  extremization  criterion. 


As  before,  then, 

r  -.-1 


'~l' 

r-ii 

y 

^11  ^12 

m 

-2 

-2 

y 

_^21  ^22  _ 

m 

+ 


V' 


is  our  original  system  model. 


(3.9) 


System  l»s  control  problem,  namely, 

gT  =  min  E 

^  -1 
m 

J.  .  -1^-1 

1^  ^11  C  » 


r'-l"  „  -1) 

[l^ 


subject  to 


=  -tA 


-1  _1 
11  ^ 


(3.10) 


has  this  solution 


For  System  2  we  have 


2h 


g„  =  min  E 

-2 

m 


subject  to 


/  =  A22:  5^+6^'  «nd  m2*  = 

Therefore,  y^  ~  ^11  ^"^11  ^12  ^“^22  ■*■ 

=  A.21  ^"^11  ^22  ^"^22  ^ 


—  21^  “I  t  A  "1 

y~t"^  -a-,9  a„„  p. 


-2^  -5 

y  =  e  -  p. 


12  "22 

-i  -j. 

A21  kjj_  ^l 


Now  for  System  3  have 


g*  =  min 

trr^ 


H. 

■  — 1  ^  —2^ 

*113^ 

®22 

3^j 

-1* 

.  .  “2* 

~2 

-1 

11 

m 

+  kj2  •♦■  A^2 

n 

+  € 

-1* 

.  .  -2*  .  . 

-1 

.  -2 

71 

m 

+  A22  ■*  A22 

+  € 

The  solution  to  (3,13)  yields 

-rl*  _  ,-l  -1 

^  ^11  ^ 

^  -1  -2 
l\  ^22  ^ 


-2^-2  -2 

^  «  —  p. 


(3.11) 

(3.12) 


(3.13) 

(3.1ii) 

(3.15) 


(3.16) 


2? 


Let  us  check  this  result  against  what  we  mi^t  have  ob¬ 
tained  bjr  solving  the  "true  optimum*  problemi 


g*  =  min  E  (y'''  B  y) 
m 


(3.17) 


subject  to  y  =  Am  +  £ 

The  expectation  and  derivative  of  g  yields 

B  (Am  +  (T)  =  0  if  a"^  B  A  is  positive  definite  and 

B  and  A  have  inverses  then  S»  =  -A~^  p 


hence 


is  identical  to  (3*16). 


and  we  can  see  that  this  result 


3.3  Model  3  Cost  of  control 

In  this  model  we  consider  a  situation  similar  to  those  in  • 
Models  1  and  2,  except  that  we  now  consider  an  extra  term  in  the 
performance  functionals  irtiich  represents  the  cost  of  the  manipulated 
variables  in. 


System  1 


gT  =  rain 

-1 
ra 

-1  .  -1  -1  -d 

^y  =  A^^  ra  4  e  »  G 


-1^ 


1^  ®11  1- 


-1 


'11 


(3.18) 


are  assuraed  to  be  deterministic  and  known 


to  all  subsystems  in  this  model.  Here  as  in  all  subsequent  models 
the  necessary  conditions  for  the  existence  of  minima,  maxima  and 
inverses  will,  a  priori,  be  assumed  to  exist.  For  the  model  above 


-1* 

m 


T  -1 
^11  ®ii  ^  • 
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Similarly  for  System  2j 


g,  =  min 

-2 

m 


-2  „ 
gy  B 


22  2 


—2  -2^  -2 

y  +  “  Cgg  » 


(3.19) 


-2#  _ 

m  =  - 


^22  ®22  ^22  ^22 


-1  .  _  -2 
*22  °22  € 


System  3 


^  T 

g*  =  min  +  (m^  +  (3.20) 

■*.  3/  B22  f  ^  ff  C22  *pf)j 

where  3y  and  ^  are  defined  as  in  (3* Hi)  with  the  m  given 
by  (3.19).  Differentiation  of  (3*20)  yields 


^21  ®22  ^21  ®11 


■"  (^2l®22  \}21^**^22  ^*  (3.21) 


=2^  .  -  =1*( 


♦  e  J  ♦  m 


-2*  - 

^  ""b 


12  ®11  ^12  '^22 


^*12®  11  [' 


.  T  „  r.  -2*  .  .  -1* 

*12  ®ii  ^12  ®  *  ^11  ® 


The  problem  of  comparing  this  mode  of  control  with  "true  optimal" 
control  is  not  without  its  difficulties.  It  is  logically  possible 

for  this  mode  to  come  out  better  than  the  "true  optimal"  control 

— i  -2  —1 

because  of  the  added  terms  in  37  of  ^22  ^ 

submatrices  A^g  offer  amplification  of  the  control 

vectors  in  a  way  irtiich  is  not  available  to  the  "true  optimal" 

formulation.  If  we  are  to  charge  ourselves  for  this  amplification 
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then  perhaps  the  appropriate  goal  functional  is 


=  7^  B  y  (^+  C  ♦  m)  where  ly.= 


'21 


'12 


In  any  event  it  appears  that  in  order  to  perform  any  quantitative 
comparison  of  these  two  different  approaches  one  would  be  required 
to  perfom  a  numerical  experiment  on  a  computer  to  determine  the 
sign  and  magnitude  of  the  performance  differenceo 


Ife  now  consider  the  comparison  of  this  system  with  an 
alternative  which  is  the  same  as  our  Direct  Intervention  case,  ex¬ 
cept  that  the  second  level  unit  is  missing.  Hence  nothing  is 
compensating  for  the  neglected  interaction  between  the  first  level 
systems ,  We  will  call  this  the  non-interacting  case.  This  system 

has  exactly  the  same  performance  functional  except  that,  of  course, 

—I.  — i# 

the  terms  involving  ^  will  be  missing.  We  note  that  the  m 

are  the  same  for  each  case.  As  a  result  of  these  observations  we 

can  easily  state  and  prove  the  follow'-'ng  theorem. 


Theorems  Lot  g  ^ 


where  y^  =  y^  + 


=  min 

=  min 


B  yg  +  n)  c 


where  A*  = 


'21 


then  gg<g^ 

Proofs:  First  of  all  t^was  chosen  in  such  a  way  that  g^  is  mini¬ 
mized.  0  is  an  admissible  control  vector.  If  0  g^*  =  g^#. 
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suppose  3  an  ^  0  which  minimized  gg  and  (t^)  >  gj^* 
but  clearly  (0)  <  gg  (|.^*)  contrary  to  the  assumed  existence  of 
either  (gg*<g^*)  hr  (h.=  0  and'  gg*  < 

Wb  conclude  from  this  theorem  then  that  in  the  case  where  we  are 
required  to  pay  for  the  variables  that  we  manipulate  that  we  can 
always  do  at  least  as  well  as  if  we  neglect  the  interaction  be¬ 
tween  the  two  first  level  systems. 

Similar  analysis  could  easily  be  carried  for  the  case 
irtiere  £  is  a  random  vector. 

Summary - Conclusions  and  Applications 

We  began  in  this  chapter  by  introducing  a  method  of  multi¬ 
level  control  called  Direct  Intervention,  In  reference  to  the 
modeling  considerations  introduced  In  Chapter  2  it  should  be  remarked 
that  in  this  chapter  the  second  level  unit  is  employing  what  we 
called  a  "Direct"  model  of  the  first  level  units. 

We  began  by  considering  an  over-all  control  problem  of  the 

formt 

1.  _g*  =  min  ^  B  y  where  y  =  Am  +  £ 

€  was  assumed  to  be  known  and  B  to  be  block 
diagonal.  Here  we  saw  that  the  multi-level 
approach  yielded  the  optimum  solution. 
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2*  Next  we  consider 

g»  =  min  E  B  y  ^  >idiere  "y  •=  Am  ■*■  € 

where  here  ?'is  a  stochastic  vector  with  known 
meant  Again  we  found  that  if  B  were  block  diagonal 
we  could  achieve  the  optimum  solution. 


3.  Here  we  had 

_T  _  _T  — 
g*  =  min  y  B  y  +  m  C  m 

subject  to  7  =  An!  +2’  where  B  and  C  are 

block  diagonal  and  £  is  a  known  disturbance.  We 

discussed  certain  difficulties  of  comparison  with  the 

•true  optimal**  case,  but  we  were  able  to  show  that 

this  mode  of  ••intervention'*  always  provided  a  better 

control  solution  than  one  could  achieve  by  neglect- 

ing  the  cross-coupling  terms. 
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U.  INDIRECT  INTERVENTION  -  STATIC  CASE 

Introduction 

In  this  chapter  our  aim  is  to  explore  a  set  of  control 
models  quite  similar  to  those  in  the  previous  chapter.  The  principal 
difference  will  be  the  method  of  coordination  employed  by  the  second 
level  unit.  The  previous  chapter  was  concerned  with  what  we  might 
call  ••direct  signal  intervention**  models.  In  that  case,  the  second 
level  unit  was  supposed  to  be  able  either  dii'ectly  or  indirectly  to 
insert  extra  inputs  into  the  3>’'stem  at  appropriate  points.  In  this 
chapter  the  second  level  unit  influences  the  outputs  of  the  system 
by  setting  the  values  of  certain  parameters  in  the  performance 
functionals  of  the  first  level  systems.  As  we  will  see,  this  method 
has  both  its  advantages  and  disadvantages.  Among  its  advantages 
are  the  following?  It  is  possible  to  effect  large  reductions  in 
the  necessary  computing  time  and  memory  required  to  compute  the 
control  solutions.  Secondly,  it  virtually  can  be  applied  to  all 
classes  of  control  problems — -at  least  all  thosi.  for  which  it  is 
possible  to  find  a  method  of  reticulation.  For  exanple,  it  is 
difficult  to  conjure  up  a  model  used  in  the  area  of  Operations  Re¬ 
search  where  the  direct  intervention  approach  could  be  applied,' 
however,  the  method  we  are  about  to  describe  can  be  applied  in 
several  situations  rather  naturally  (see,  for  example.  Chapter  8)« 

Among  the  disadvantages  of  this  approach  is  that  a 
complete  closed-form  soluti.on  is  someidiat  more  difficult  than  the 
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models  of  Chapter  3*  Secondly,  it  generally  reduces  the  con¬ 
trollability  (see  Appendix  D)  of  the  system  and  this  can  hurt 
the  performance  (this  problem  will  be  discussed  in  more  detail  in 
section  )4«3)  of  the  system  as  measured  ty  the  performance  criterion 
of  the  second  level  unit. 


In  short,  models  to  be  discussed  in  this  chapter  are 
inspired  by  an  attempt  to  trade  off  controllability  and  possibly 
performance  quality  against  the  cost  of  calculating  the  optimal 
control  vectors. 


U . 1  Model  1 


Again  we  start  with  two  first  level  units. 
System  1.  ^ 


g  *  =  min 
-1 


E 


(U.i) 


-1 


where  ^  and  are  defined  as  in  Chapter  30 

The  ^’s  are  assumed  to  have  fixed,  known  means  and  finite  second 
moments.  is  a  scalar  parameter  controlled  by  the  second  level 

\init.  It  is  also  assumed  that  the  first  level  units  do  not  anti¬ 
cipate  the  values  of  p  given  by  the  second  level  unit,  i.e.  they 
wait  for  the  values  of  p  to  be  transmitted  before  the  values  of 
the  maninulated  variables  are  set.  1  is  the  vector  jjL,  1,  ...  ij 
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Similarly  for  System  2t 


2^  =  A22  ^  ^  t 


g  *  =  min  E 
^  _2 
m 


•  i  •  »*) 


The  optimal  control  vectors  for  the  first  level  units  are 


J.# 

m 


-2*  _ 

m  =  - 


^11  ®11  ^11 


-1 


-t  -1 


^22  ®22  ^22 


iriiere  =  E  ) , 


1'.- 


T 

'll  "'ll 


T  +  A-,-,  B,,  p 


i  *  A'J 


‘} 

} 


(U.2) 


Ci^-3) 


Now  for  the  second  level  unit  we  pose  the  task  of  manipu¬ 
lating  the  parameters  and  in  such  a  way  that  the  neglected 

interaction  (via  -^21^  least  partially  compensated. 


C  is  a  vector  of  fixed  quantities. 


CU.U) 


-1  _  .  -1*  ,  .  -2*  .  3-1 

y  =  A^^  m  +  A^2  ®  ^  ^ 

-2  .  J.*  .  —2*  -2 

y  =A2im  +A22m  +  £ 


using  (U*3)  '*»ith  (U.U)  and  (ii.5)  we  have 
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3*  <>11  ^2  <>12  'hi  *  h2  *  ti 

®11  \fl  <>11  +  ^2  <>12  •"  ^11  ■♦'  ^12 

12  ■*■  ^11  *  ^12  '*’  ^ij  ®12  pi 


*  2 


[a  *11  *  ♦ 

*  ^21  *  P22  *  e?] 

♦  i/itai*  /^2  *2!;*  »ai*  ^<22*  f 


1  <>21  •*'  ^2  ^22 


/ 


■2  ®22  /l 


<>21  ■*■  ^2  <>22 


_  ^^21* 

5^22  -^  £2 

^11  ®11  ^11 

-1  >  -> 

«  1  - 

^11  ®ii  h-r 

—  - 

-1 

-/’2,f2- 

T 

^22  ®22  ^2_2 

•  T  -  (Tg 

A22  B22  A22J 

■where 

cj,  c 

2  =  Ifk  +  1 

<>11  “  ^11 

^11  ®11 

-1 

•  1 

^11  ^  *^11 

^  -1 

'  T“^  T 

-1 

=  -Bii 

All  •  1 

-  -T- 

r  T 

-1-1 

<>12  "  “*-12 

[_^22  ®22  ^22_ 

] 

’^12  '^12 

<>22  =  -B22^ 

_lT  „ 

A22  °  1 

>22=^ 

<M 

f 

it 

H 

CM 

%i  ®ii  -^i; 

r-^ 

7*21  "  ■^21 

-1 


-1 


T  -P 

*22  »22  f 


••»nj 

[*n  ®u  *:i^ 


-1 


All 


»1  -2 


-1  -1 


B  is  not  necessarily  block  diagonal  in  this  model  but  it  is  assumed 
symmetric  and  positive  definite o  After  some  painful  but  simple 
manipulations,  one  could  take  the  expectation  and  the  derivative 
of  (U<.6)  -with  respect  to  and  and  equate  them  to  zero  to 


3h 


find  the  optimizing  veluea  of  p  ^  and  ^  2°  ^  brief  inspection  of 

(U«6)  shows  that  one  would  get  two  equations  of  the  following  form: 

•1  1^1*  *  *2  A*  *  *3  =  0  (‘*•7) 

»!  A*  *  »2  /2*  *  ° 

This  can  be  solved  for  p  if 

We  could  now  substitute  and  back  into  g^* 

and  compare  this  with  several  possible  alternative  control  modes  and 
for  several  classes  of  disturbance  vectors.  However  at  this  level 
of  complexity  this  sort  of  work  is  probably  best  left  for  numerical 
!  experimentation  on  a  digital  computer. 

We  can,  however,  compare  this  control  mode  with  the  case 
where  the  system  is  reticulated  but  the  interaction  between  the 
reticulated  parts  is  completely  neglected.  This  case  differs  from 
the  model  just  considered  in  that  no  second  level  unit  is  considered 
and  hence  ^  =  1. 

In  the  model  just  considered  assume  that  with 

probability  1  (deterministic  disturbance).  Let  gjJ*  be  the  optimal 
value  of  the  system  performance  functional  for  the  indirect  inter¬ 
vention  approach  (Model  1).  Let  Ejjj*  be  the  optimal  value  of  the 
system  performance  functional  for  the  case  of  neglected  interaction. 


3^ 


Theorem  U*l« 

Proof i  Let 

/l  ^’2  "  ^ 
But  f  (1>  Ij  £ )  — 


5  %!* 

are  admis sable  values  of  the  control  variable B. 
Therefore,  since  f  ^2*’  ^ 


l,e) 


Next  we  will  prove  a  theorem  which  illustrates  the  difficulty 
of  comparing  two  control  modes  on  an  absolute  basis  when  the  input 
disturbances  are  stochastic.  We  can  view  the  processes  we  have 
treated  as  being  control  problems  which  arise  at  regular  intervals 
in  time  but  each  interval's  outputs  and  performance  functionals  are 
statistically  independent  of  those  in  preceding  intervals. 


Theorem  U.2. 


Let 


W 


for  i  6  I  (1  is  a  denumerable 


index  set)  be  a  vector-valued  independent  random  process.  Let  y^ 
be  the  output  variables  associated  with  the  sequence  of  control 
vectors  nij,  .  Let  yj^s  ■  sequence  of  performance 

functionals  for  i  £  I  .  Consider  two  different  control  modes  yield- 

-  1 

^i  ’  “i 


—  1  —  1  •2—2 

-  and  y^  ,  respectively.  Assume  there  exists  a 


ing  y.  ,  m 


sequence  for  k  €  I  such  that  P  /gj^ 


?',j)  >  oj  »nd  furthw  Th«i 


12 


almost  surely  for  an  oO  niunber  of  points  i  G  I  the  performance  of 
control  mode  1  will  exceed  the  performance  of  control  mode  2. 
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The  proof  follows  directly  from  the  Borel  CentelU  LemiMS.  See, 
(Q\ 

for  example,  Doob.'  ^ 


Essentially,  then,  if  it  is  possible  for  one  control  mode 
to  be  better  than  another  (for  its  utility  functional  to  take  on 
a  hi^er  value)  and  further  if  the  sum  of  the  probabilities  of  this 
event  taken  over  all  time  is  unbounded  then  with  probability  1  this 
event  will  occur  not  once  but  an  infinite  number  of  times «  However, 
it  is  still  possible  for  one  control  mode  to  be  better  than  a  second 
on  the  average  and  still  have  the  second  better  than  the  first  for 
an  infinite  number  of  points  in  time.  We  have  stated  this  theorem 
to  point  out  the  difficulties  involved  in  comnaring  two  control 
modes  on  an  absolute  basis  irdien  the  system  being  modeled  is  subject 
to  probabilistic  disturbances. 


li,2  Model  2 


We  next  consider  a  model  where  the  "cost  of  control**  enters 
the  performance  functionals  as  a  quadratic  term  instead  of  linearly 
as  before. 

System  1 


=  min 
m 


®ii  1^^ 


(U.8) 


subject  to  ^y^  =  m^  -f  , 

In  this  model  the  terms  are  considered  to  be  deterministic 

and  known.  The  method  of  attack  for  probabilistic  disturbances  has 
been  outlined  in  previous  models. 
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System  2 


g  *  =  min 
^  -2 


’22 


-2 

2^^ 


-2 

m 


'22 


~2 

m 


(U.9) 


m  ~ 

-2  -2-2 
subject  to  2^  =  A22  m  +  £  . 

Agein  the  quantities  ^ ^  are  acost**  variables  manioulated  by  the 
second  level  unit  (System  3)«  From  the  point  of  view  of  the  first 
level  systems  the  are  fixed  parameters,  i.e.  they  do  not 
anticipate  or  predict  values  for  these  quantities  but  rather  wait 
for  the  second  level  unit  to  communicate  this  information  to  them. 


The 

optimal  values  for 

m^  and 

-2 

m 

are 

given  by 

-1* 

m 

[^n  ®11  ^11 

‘^ll 

-2« 

m 

[^22  ®22  *22 

^^22 

/2 

T 

*22  ®22 

(U.IO) 


N<Jw  for  the  second  level  unit  we  haves 


=  ndn  \  ^  B  ^  +  m*"^  C  m» 


'^l’^2 


■} 


(li.ll) 


m*  = 


-1* 

m 

-2* 

m 


C  = 


Cii  0 


0  c 


22 


B  = 


Bn  0 


0  B 


22 


r  -11 

-1 

-I*  . 

.  -2* 

3^ 

m  + 

Ai2  m 

+  e 

-2 

y 

-2  _  . 

-2* 

3^  J 

37 

m  + 

A22 

€. 

Again,  the  second  level  unit  attempts  to  set  in  such  a  way  as 
to  compensate  for  the  interaction  terms  neglected  in  the  models  of 
Systems  1  and  2. 
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Naively  we  mi^t  substitute  (U.IO)  into  (li.ll)  »nd  attempt 

to  carry  out  the  minimization  analytically.  However,  it  is  here 

that  we  begin  to  run  into  difficulty;  for  and  are  contained 

in  the  inverse  of  a  fairly  complicated  matrix  and  hence  enters  g^* 

in  a  way  >rtiich  is  quite  difficult  analytically.^  We  will  employ 

f  211 

the  following  theorem  from  Taylor'  ’  to  remedy  this  state  of  affairs. 

Let  be  the  space  of  all  bounded  linear  operators  defined 

on  the  Banach  space  T.  mapping  X  into  itself. 


Theorem  U.3. 
where  | | A | | 
tinuous  and 

of  7.  -  A. 


If  A 


e[x] 


and 


1^1 


A)  exists  and  is  con- 


is  the  norm  of  A — 

(X“  A)  ^  y  =  ^  A*^"^  y  for  each  y  in  the  range 

1  CtO 

Further  since  X  is  complete  (5  -  A)  ^  ^  ^  A*' 

1 


(See  Appendix  (A)  for  definitions  of  the  terms  employed  in  this 
section) . 


Since  we  have  assiimed  that  the  vectors  y  -  C  ,  m  are  de¬ 
fined  over  Euclidean  n-space,  the  conditions  of  the  theorem  regarding 
A  are  satisfied. 

Now  we  return  to  the  problem  of  finding  a  tractable  expression 
for  m^*  and  ra^*. 


In  numerical  treatments,  though,  this  is  not  particularly 
alarming.  If  all  of  the  numerical  values  of  A,  B,  C  and  are 
specified  we  have  a  fairly  simple  search  problem  in  two  variables. 
However,  we  would  like,  if  possible,  to  offer  an  approximation  theory 
for  such  problems. 
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m^*  =  - 

[^11  ®11  ^11 

^^11 

1 

®11 

'  -  [*n 

®11  ^11  °11 

1  -1 

^11  ®11 

Let  T]_  =  - 

®11  ^11  ®11 

V  '  -  'll’'  \f^  -  ’■ij  *n  *11 


Similirly  “  '^2]  ^22  ®22  ^ 

T  -1 

where  Tg  ~  -  ^22  ®22  ^^22  *^22  * 


(k.l2) 


(h.13) 


Now  for  >  IIt^M 

Pi  -  ’1]  ■'  “ 


we  can  invoke  theorem  It.. 3  to  express 
.nd  ^ 


We  can  now  truncate  these  series  at  any  point  depending  on  the  de¬ 
sired  degree  of  approximation  and  using  the  resultant  apnroxiraations 
for  (1;*12)  and  (U.I3)  we  could  solve  (li.ll)  analytically.  As  can 

be  easily  verified,  the  truncation  of  the  series  above  at  the  first 

^  1  ^  T 

terms,  i,e.  -  T^)  =  —  and  -  Tg)  =  -  yield  a  set 

^2 

of  simultaneous  linear  eqiiations  for  the  optimal  values  of and  ^go 


The  approximations  developed  above  are  valid  for 
|^(  >  On  the  other  hand  for  small  we  can  approximate 


Pi  -Tj  by  -li-l 
for  both  large  and  small 


and  hence  we  have  a  valid  approximation 


For  a  qualitative  comparison  of  this  model  with  the  case 
where  the  interaction  between  first  level  units  is  neglected  we 


Uo 


invoke  theorem  li.l.  The  proof  must  be  slightly  modified  to  account 
for  the  change  in  the  nerformance  functional  from  model  Hence 
we  can  say  that  no  matter  Tdiat  the  (deterministic)  disturbances  £ 
are  the  indirect  intervention  approach  yields  better  performance 
than  the  one  obtained  by  neglecting  the  first  level  interaction. 

It. 3  Model  3 

In  the  previous  model  we  considered  indirect  intervention 

in  the  case  ’where  the  "intervention  parameter'*  was  inserted  in  the 

term  Trtiich  yields  the  "costs"  of  maninulated  variables,  W®  now  wish 

to  insert  the  "intervention  parameter"  in  the  term  of  the  functional 

— T  — 

which  sets  the  "cost"  of  y  By. 


System  1 


g  *  =  min 


-1  L 

m 


17  B 


-1 


11  r 


-1" 


—1  —1  —1 

subject  to  ^y  =  m  +  £ 


System  2 


g  *  =  min 

-2 

m 


27  Bgg  ^  +  m. 


—2  -2—2 
subject  to  27  =  A22  m  +  e 


-i 


The  derivative  of  with  respect  to  m' 


4  ^ii  ®ii  (^ii  ^il  =  0 


yields 

and  if  we  let 


(li.l3) 


(U.lb) 


Ui 


we  find 

=  -  [*U  'll  *11  ‘  'll  ^±1  [hi  'll 
If  we  look  b»ck  we  see  that  (U*l5)  has  exactly  the  same  form  as 
(liolO)  and  hence  we  need  go  no  further.  Unless  there  have  been 
special  restrictions  placed  on  the  f  this  form  of  intervention 
is  the  same  as  that  in  Model  2, 

U.ll  Indirect  Intervention 

Let  us  consider  briefly  what  precisely  is  involved  in 
what  we  have  called  "Indirect  Intervention".  For  a  single  system 
or  subsystem  lot  us  consider  a  2n  +  1  dimensional  "system  space", 
and  let  the  first  n  a?d.s  I'epresent  the  control  vector  components 
m.  Tfs  will  call  this  the  "control  subspace".  Along  the  next  n  axis 
are  measured  the  disturbance  components  €»  This  is  the  "distwbance 
apace".  Along  the  last  axis  is  measured  the  "quality"  of  perfor¬ 
mance  as  measured  by  the  performance  functional.  Therefore  it  is 
assumed  that  if  one  picks  a  pqint  in  the  "control"  subspace  and  a 
point  in  the  "disturbance"  subspace  that  a  particular  value  of  the 
"quality"  component  is  determined  thereby. 


We  note  that  in  the  previous  problems 


U2 


Now  wiwin  these  vectors  were  substituted  into  we  had 
g_*  =  min  S-i  (/S  It  4  o*  ^)*  Therefore  where  we  began 


with  a 


problem  in  n  controllable  variables  we  now  have  a  problem  in 
only  two  manipulated  variables.  The  questions  we  would  like  to 
answer  ares  'Tfhat  is  the  geometric  countemart  of  this  process?" 
and  '"What  did  we  lose  in  quality  of  performance?"  The  answer  to 
the  last  question  depends,  of  coui’se,  on  the  specific  system  and 
the  particular  disturbance  vector  £•  The  first  question,  however, 
can  be  answered  as  follows j  We  have  introduced  a  two  parameter 
family  of  curves  into  the  "control  subspace"  and  the  resultant  con¬ 
trol  vector  must  now  lie  somewhere  on  this  surface.  Fig.  U.l 
describes  a  simple  case  with  two  m  control  componarits  and  a  single 

Z’. 


Where  originally  any  point  in  the  m^,  m^  plane  was  an  admissible 
control  vector,  now  only  those  points  lying  on  the  one-parameter 
curve  K  are  admissible.  Therefore  we  traded  an  optimization  problem 


U3 

in  two  variables  for  one  in  a  single  variable  but  we  radnced  the 
set  of  admissible  control  vectors  in  the  process. 


Summary 

In  this  chapter  we  introduced  a  new  method  of  seoOndL  level 
"coordination"  and  we  Investigated  its  properties  by  employing  it 
in  three  different  models. 

Model  1 


First  Level  Units  g^*  = 


subject  to  y^  =  m^  ^ 


=  min  ^  *  2  •  m^J" 


i  =  1,  2 


Second  Level  Unit  g^*  = 


where  y  =  Am  •  ♦  ? 


=  min  El  y**^  B  y  +  m  -  f 


Model  2 


T  T 

First  Level  units  g^*  =  min  (y^  y^  ♦  ^  m  ) 


-i 
m 

-i  ^  -i 

T  =  A.,  m  + 


subject  to  y  =  A^^ 

Second  Level  Unit  g.*  =  min  (y^  B  y  4.  m  C 


i  =  1,  2 


where  y  =  Am  +  S 


Model  3 


First  Level  Units 


where  y  =  A 


ii 


g.*  =  min 

^  -i 
m 

-i  ^  -i 
m  ♦  £ 


^  B 


ii 


-i  ^  J 
y  4.  m 


C 

l^ii  m 


i  =  1,  2 


Second  Level  Unit  (Ssme  »s  Model  2) 


Conclusions; 


1«  For  Models  1,  2  and  3  with  deterministic  distur¬ 
bances,  indirect  intervention  always  yields  better 
performance  than  that  achieved  by  neglecting  the 
interaction  between  the  first  level  systems o 

2.  Indirect  Intervention  reduces  the  controllability 
of  the  original  s:y3tem  by  reducing  the  class  of 
admissible  control  vectors. 

3«  It  is  irrelevant  whether  the  second  level  unit 
fixes  the  "price**  of  the  control  vectors  or  whether 
it  sets  the  penalty  "cost**  associated  with  the 
outputs.  Both  models  of  control  are  equivalent. 

U,  It  is  usually  impossible  to  compare  the  perfor¬ 
mance  of  alternate  methods  of  control  on  an  absolut 
basis  when  the  systan  is  subject  to  stochastic 
disturbances. 

Applications 


See  Appendix  (C) 


( 


DIRBCT  INTERVEWTION  -  DYNAMIC  CASE 


Introduction 

We  will  leave  the  domain  of  statics  and  consider  now  a 
class  of  multi -variable  dynamical  systems.  In  particular,  our 
representation  of  the  system  dynamics  will  be  in  discrete  time.  The 
basic  dynamic  model  will  be  the  following: 


^t  +  1  ^  ^  ^t  ”t  4.  1  ^t  4.  1»  ^0 


=  C  t  =  0,  1, 


The  quantities  ^y^^  are  n-component  vectors  of  state  variables. 


A  is  an  n  X  n  non-singular  matrix. 


of  manipulated  variables.  The 


{”t} 


(5.1) 


■component  vectors 


is  an  n-dimensional  vector  valued 


random  sequence  with  known  means,  i.e.  E  assume 

T 

S  (€^6^)  =0  if  t  ^  and  E  (6^  ’’  -<r  oo . 


Mathematical  models  such  as  (?.l)  may  be  taken  to  mean  that 
significant  changes  in  the  system  take  place  only  at  evenly  spaced 
intervals  in  time.  Alternatively,  we  may  interpret  (?.l)  as 
observations  of  a  continuous  system^  taken  at  discrete  intervals 
of  time.  The 
intervals . 


are  assumed  to  be  constant  in  the  inter-sampling 


Similarly  the  presence  of  random  disturbances 
viewed  in  two  different  ways.  We  mi^t  assume  that 


can  be 


The  behavior  of  linear  systems  in  the  inter-sampling  period 
has  been  explained  by  control  theorists.  (11) 
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1  ~  ^  “t  4  1  (^*2) 

l8  an  exact  model  of  the  !?yat«m  dynamics  but  that  certain  external 
influences  affect  the  state  variable  and  are  accounted  for  by  the 
term  ,  in  Alternatively,  we  may  asstime  that  (5«2) 

is  ain  inexact  model  of  the  real  system  and  hence  the  terra  4.  ]_ 
accounts  for  the  difference  between  the  predicted  values  of  the 
state  variables  and  those  that  actually  occur.  We  assume,  in  both 
cases,  that  the  (£'a)  are  observable  after  the  fact,  i.e.  at 
time  T  =  T  +  1,  1  •••  ^3^  »re  all  known. 


Tfhile  it  may  apnear  at  first  glance  that  a  model  such 
as  (5ol)  is  hi^ly  restrictive  and  specialized,  certain  techniques 
of  substitution  can  be  used  to  reduce  any  model  of  the  form 


yl 

^t  +  1 


=  ^1  ^t 


**'^k^t-'k  +  l^®'”t  +  l*^^t  +  l 


to  (^.1)  by  a  suitable  relabeling  of  the  variables. 


The  over-all  system  optimization  problem  isj 


g»  =  max  E 


ni- « « 

1  n 


t  =  1  ^  ^ 


(^•3) 


subject  to 8 


^t  +  1  "  ^t  *  ”t  +  1  ^t  +  1 


1 


5,1  Model  1 
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As  in  Chapter  3  *nd  U,  we  construct  the  two  first-level 
control  problems  as  follows* 


System  1 


f  N 


*1* = 'll  i?i  -  >=11  =t}  (s-**) 

2^  rafera  to  Syatem  I'a  model  of  ^  L^l  ^2  C^)]‘ 

l^*l'*lll5^-*\*l*8iel'  1?^=  (5-5) 

Similarly  for  System  2* 


g,*  =  ^  *2”“  V 

r  N 

Ct  =  1 

2^t  ®22 

-2  -2"^  -  -2  1 
a^t  -  \  <^22  ”t  J 

•  (5.6) 

-2  _  .  -2 

.  -2 

-2 

-2  -2, 

(5.7) 

rt  1~  ^22  2^1 

"t  +  1 

’*'€t  +  l> 

2^0  '  ® 

Now  we  form  a  second  level  system  whose  fraction  is  to  compensate 
for  the  neglected  interaction  between  the  two  first  level  systems. 


g3*  = 


{NT  T  ''i 

3^t  ®113^t‘"\t  ^11 

t  *  1 


3^t  +  1  ^11  ^t  ^  ^12  ^t'*'"t4.i'*'  ^t  +  l*  ^4.1 


and 


V'  = 

«max_2 

>'fc 

®22 

‘^1, 

N 

(t  =  1 

-2 

.  J. 

,  -2^-2* 

ot  “2 

h^t  4 

1  " 

A21  y^  + 

^22  ^t  ■*'  "t  4 

1  ^  'it 

It8 


It  is  important  to  note  that  in  this  representation  the  second  level 

control  ' action  is  decomposed  into  the  determinations  of  two  functions, 
-1*  , 

^t 

necessary  to  ignore  certain  aspects  of  the  system  dynamics  in  order 
to  avoid  having  the  second  level  unit  solve  a  problem  of  the  same 
dimensionality  as 


•  In  order  to  perform  this  split  successfully  it  is 


We  assume  that  at  time  t+1  the  sample  value  of  the  outputs 
y^  are  known  or  measurable  by  every  subsystem  without  error.  We 
visualize  that  at  time  t+1  the  operation  of  the  decent ra,li zed 


system  proceeds  as  follows t 

_1  o 

1.  Systems  1  and  2  measure  y^  and  y^  respectively. 

2.  Systems  1  and  2  optimally  determine  ^  ‘nd 

^  ^  respectively. 


з. 

и. 

6. 


-1* 

"^t  +  1 

Second  Level  Unit  (System  3)  measures  yf. 
System  3  computes  using  2. 

System  3  computes  ^  using  ^  1  “id 


7.  All  decisions  are  implemented. 


Hence  when 


system  3  determines 


'It 


+  1 


it  measures 


and  uses  it 


in  the  optimization;  thus  it  completely  neglects  the  Affect  that 
— i 

^  will  eventually  have  on  ^  similar  remark  applies 


to  the  determination  of  ^ 


h9 


1 

I 

I 

I 

I 

m 

I 

I 

I 

I 

I 

I 

I 

1 

1 


We  now  proceed  to  solve  the  four  optimizetion  problems 
in  Dynamic  Programming  formalism.  For  System  1  we  let 


T  J.  j. 

11  1^1  -  ”1  °11  “l 


T 

(^)  *=  max  Ejj^B 
”1 

^  -  1  ^^11  ^  ^  ^)J' 


(5*10) 


1 


where  (c  )  is  the  maximum  expected  ^return"  for  the  N  stage 
process  given  the  process  starts  with  y^  =  ^.  ^y^  is  given 

by  (5.5). 

Similarly 

I  2  ^  o  o 

Sjj  (c)=m«  S  ®22  2^1  ^  °22  ”l 


^y^  is  given  by  (5.7). 


For  Sj^stem  3  we  have 


-  3  /-lx  _ 

V  (c  )  =  max 


^j^yi  ®11  3^’!  "  Y\1  ^liy^l 


*  ®N 
is  given  by  (5.8). 


1 


1 


S„^  (^)  ®  E 

'U 


[3^ 
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T 


-2  —2^  -2 
®22  3^1  ”  ^1  ®22  ^1 


(5.13) 


i.**- 1  (/i>} 


is  given  by  (?,9). 


_1 


Note  that  in  each  case  the  actual  influence  of  ^  2^ 


on 


0  o  n 

^  2;^,  and  0^  4.  2.  ^t  •♦■  1  neglected.  We  also  note  that 


if  in  Sjj^  and  we  let 


*  1  *  1  *  *12  ^  -eL  1- 

et .  1  *  1  *  *21  »k  i> 


(5.1U) 


then  all  four  Dynamic  programming  problems  have  the  same  structure, 
i.e.  the  variables  in  each  pair  of  problems  are  isomorphic.  Hence 
all  are  mathematically  equivalent  to  the  problem 

B  y  1  -  C  Sj.  (5.15) 


_  1  (A  c  +  «!  +  j. 


y^  =  A  c  ♦  Cl  ^t  +  1  "  A  y^  *  m^  ^  ^  ^  2. 

The  general  solution  for  is  of  the  following  form 
(See  Appendix  B). 

(5.16) 

where  c  is  current  state 


=v*-  k  =  -  k  *  =  *  'V  - 


of  the  system 
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®N-k’  (^*17) 

^N-k“^®"®N-k'^  ^N-l^N-k'*'^  ®N-k  +  l]  (^•IS) 

-  k  =  Wn  -  k)^  ®N  -  k  (T  +  »N  -  k>  -  V  -  k  °  »N  -  k 

Sn  -  k  =  ^  «N  -  k)"*  Sn  -  k  -  k  -  k)  -  V-  k  C  On  -  k 

Hjj  =  (C  -  B)“^  B  ®N  "  ® 

Sn  +  1  =  ° 

Therefore 

\  -  k  ^  ^N  -  k  ^11  ^N  -'  k  -  1  '*'  ®N  -  k  (?»23) 

-2*^„2  ,  -2 

”n  -  k  -  k  ^^22  -  k  -  1  ^N  -  k 

is  identic*!  to  Hjj  _  in  (5*  17)  except  that  A - 

B - . ,  C - ^C..,  ^ ( - ^eans  wis  replaced  by”) 

XX  xx  • 

Similarly  for  Qfj^_ 


For  /j/]; 

,  and 
k 

A  *2 

-  k 

we  have 

-  *1 

„3 

-11 

^N  -  k 

■  ^  -  k 

-  k 

M 

1 

+ 

1— 1 

1 

—  *2 

.fU 

-2 

k 

Si  -  k  ■ 

■  ”n  -  k 

^N-k 

-  1  ^  -  k 

C5.2U) 
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3  1 

Hjj  _  ia  identical  to  _  jj.  except  that  S— — and 
T 

^ ).  Similarly  for  ^  and  ajj^_  ^  and 
Therefore  the  true  outputs  of  the  system  arer 

^t  +  1  -  All  ^t  •"  Ai2  yt  ''’^t  +  i.-"^  +  1  ■"  \  +  1 

—2  -JL  -2*  -2*  -2 

idiere  the  m»a  are  given  in  (5.23)  and  the  tl'a  in  (5.2li). 

Now  that  we  know  the  specific  form  of  the  deciaion  rule 
we  can  conceptually  comiiare  the  performance  of  g  as  given  by 
(5.3)  with,  for  example,  as  given  by  (5.8)  and  (5.9). 

These  quadratic  forms  become  random  variables  whose  distributions 

_  (ij) 

are  determined  by  the  joint  distribution  of  the  £’s.  Middleton^  ' 
has  shown  that  if  the  a  are  generated  by  an  independent  gaussian 
random  process  then  any  positive  definite  quadratic  fora  in  the 
?''a  is  distributed  by  the  Non-Central  Chi-Squared  distribution. 
Approximation  techniques^^*^^  have  been  developed  which  reduce  the 
Non-Central  Chi-Cquared  distribution  to  the  Normal  distribution  and 
since  the  sum  and  difference  of  Normal  variates  is  again  a  Normal 
variate  we  can  find  the  P(g-g2~g^  >0),  i.e.  that  decentral¬ 

ized  approach  yields  a  better  value  of  the  performance  functional 
than  the  "centralized”  or  "integrated"  approach. 


^3 


A  csreful  ex»min*tion  of  equations  (5»U)  through  (^•9) 
brings  out  an  interesting  conclxisionj  if  we  modify  the  first  level 
performance  functionals  appropriately  then  the  second  level  unit 
does  not  appear  to  perform  a  useful  function j  in  this  sense,  then, 

system  is  equivalent  to  a  2^  —  system.  This  is  due  to 
the  fact  that  the  two  functions  assigned  to  the  second  level  with 
a  view  to  reduction  of  the  dimensionality,  can  as  well  be  located 
in  the  first  level  unitso 

^,2  Model  2 

In  the  models  previously  considered  we  have  assumed  that 
the  performance  functional  was  separable,  i.e,.  Block  diagonal, 
but  that  the  "system'*  matrix  A  was  not.  In  this  section  we  will 
demonstrate  a  method  which  transforms  the  original  problem  (5«3)  into 
one  where  the  "jystem"  matrix  is  diagonal  and  hence  separable. 
Unfortunately,  the  conditions  for  separability  of  the  transformed 
performance  functional  are  rather  restrictive,  W*  note  that  if 
both  the  performance  fractional  and  the  A  matrix  are  block  diag¬ 
onal,  i,e,  separable,  then  the  over-all  problem  can  be  decomoosed 
into  two  or  more  comnletely  separate  problems.  In  our  language  this 
means  that  the  first  level  units  can  work  completely  separately 
and  the  over-all  system  performance  will  not  suffer  as  a  result. 


We  begin  by  considering  (5*3 )j 


i 


g* 


max 

m^* • tO 


<!"ij  '5.26) 


=  c 


idiere  ^t  +  1  ^  ®t  +  1  ^t  ♦  1 
We  will  assume  for  this  section  that  the  eigen  values 

are  all  distinct  and  that  none  are  zeri04  Hence^^^  there  exists  a 


t'} 


of  A 


transformation  S  such  that  S'^  A  S) 


=A. 


where 


X, 


Therefore  from  (^.26)  we  have  ^  j.  =  A  8  S^’  7^  \  4.  1 


»T  =  .  T  - 


.^*0  0  .. 

1  0  ... 

1 

0  •  •  •  • 

and  further  3  =  I  * 

0  1  ...  0 

1 

• 

:  0  •«« 

0  .....  1 

- 

0  0  ...a; 

• 

• 

T  — 

®  ^t  +  1 


,T  - 


and 


Now  let  yt  +  1  =  S  y^  ^  ^  ^ 

^  X  ~  ^  ^t  +  1*  Therefore  the  oerformance  functional  becomest 

^  r  '  ' 

g*  »  _g  max_g  E  ^  is  y^f  B  (S  ^)  -  (S  m®)’’  C  (5.2?) 

®1>  • '  t  “  1  ^ 

(S  <)j- 


=  max  E 


tk' 


T 

8^  B  8 


y®  -  3®  8^  C  8  5^ 


t  =  1 


,(5) 


Now  it  is  known  that  if  a»  B,  and  C  are  simultaneously 


permutable,  i.e.  if  A  B  =  B  Aj  A  C  =  C  A,  etc.  then 

B  S  = 

In  this  case  then  we  can  split  the  over-all  problem  not  into  two 
subproblems  ydth  several  optimization  variables  in  each  but  into 
n  single  variable  problems  each  of  which  is  completely  independent 
of  the  others.  Although  the  aforementioned  conditions  are  rather 
restrictive  they  are  well  worth  examining  because  of  the  complete 
separability  which  results. 

Summary  and  Conclusions 
5.1  Model  1 

A  class  of  system  models  which  possess  significant  dyna¬ 
mics  were  formulated  and  the  method  of  Direct  Intervention  was  dis¬ 
cussed  as  a  method  of  coordination  for  the  second  level  unit  in  a 
2 “  3^  system.  The  associated  analytical  work  involving  the  solution 
of  a  Dynamic  Programming  problem  was  carried  out.  The  Dynamic  Pro¬ 
gramming  solution  represents  a  slight  extension  of  the  work  of 
Adorno, The  possibility  of  the  comoarison  of  this  decentralized 
control  system  with  the  wintegrated"  approach  were  discussed  and 
the  analytic  difficulties  noted.  Approximations  were  discussed 
that  could  be  used  for  comparative  purposes  if  the  system  parameters 
were  known.  In  conclusion  it  was  noted  that  this  method  of  inter¬ 
vention  leaves  much  to  be  desired  in  that,  except  possibly  as  a 


0  0  .  ( 


0 

0  0  • 

« 

0  Q  O  0  •  0 


0  0 


•  b  . 

n, 


and  C  S 


-JV.. 


% 


device  for  the  division  of  computational  labor,  the  second  level 
unit  serves  no  useful  purpose  and  that  this  2j2  -  ^system  is 
equivalent  to  a  1/  -  2^8(ratem. 

5*2  Model  2 

Sufficient  conditions  were  discussed  for  the  existence  of 
a  transformation  which  completely  separates  the  problem  of  Model  1» 
The  resultant  control  ontimization  equations  can  be  solved  one  by 
one  independently.  In  an  organizational  context  the  system  can  be 
separated  into  n  independent  operating  divisions  without  the  need 
for  a  coordinating  unit  at  the  second  level. 

Applications 


See  Appendix  (C) 
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6.  INDIRECT  IWraiVEJJTION  DYNAMIC  CASE 

Introduction 

As  we  indicated  in  the  introduction,  there  are  two  critical 
problems  in  optimization.  The  first  is  the  problem  of  many  state 
variables,  which  we  have  attacked  in  the  preceding  chapters.  As 
we  saw  in  the  previous  chapter,  the  counling  within  the  over-all 
system  dynamics  may  be  of  such  a  nature  that  it  is  impossible  (at 
least  with  the  models  we  have  been  considering)  to  recouple  the 
system  with  a  second  level  controller  without  considering  all  of 
the  state  variables  simultaneously.  For  these  cases  we  demonstrated 

alent  to  a  2^  -  3^  system. 

We  are  faced  with  essentially  the  same  problems  in  attempt¬ 
ing  to  apply  Indirect  Intervention  from  the  second  level  in  the 
dynamic  case.  Our  principal  effort  in  this  chapter  will  be  based 
on  an  attempt  to  reduce  the  search  effort  required  to  find  the 
extremizing  values  of  the  variables  given  that  it  is  not  feasible 
to  reduce  the  number  of  state  variables. 

The  number  of  state  variables  alone  is  not  the  sole  cause 
of  unfeasibility  in  the  computation  of  optimization  problems.  The 
problew  of  search  for  the  minimum  or  maximum,  as  the  case  may  be, 
is  also  critical  in  determining  the  cost  of  computation. 

Consider  a  problem  in  U  state  variables  and  let  us  assume 
we  want  the  solution  (numerical)  to  a  problem  of  the  following  form. 


that  a 


1^-  2^  system  was  equiv 
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(®j_»  Cg-*  o.j  Cl  )  =  min  Jg  {m^,  m.*  m.,  mi  )  (6« 

m^,m2,m^,Ej^  L 


1) 


The  c.  are  state  variables  and  =  T/  v  are  trans¬ 


formations  i^ich  act  on  the  state  variables.  The  particular  trans¬ 
formation  is  determined  by  m^  ...  m^  «  We  assume  f^  (c^,  Og,  c^, 

Cj^)  =  0.  Assume  that  the  c^  i  =  1  ...  U  can  take  on  10  values 

each  and  m^  ...  m^  can  take  on  10  values  each.  The  10  values  of 

each  of  the  m^  determine  a  grid  in  the  Cartesian  product  space 

Ml  X  Mg  X  M3  X  idiich  has  10^  points.  If  we  wish  to  find  the 

minimum  by  searching  this  grid  we  must  evaluate  the  right  hand  side 

of  (6.1)  10^  times  for  every  value  of  c^,  Cg,  O3,  or 

10^  •  10^  =  10^  times  for  every  value  of  N*  On  the  fastest  digital 

computer  this  would  take  on  the  order  of  10^  seconds/N.  We  will 

show  that  by  using  Indirect  Intoinrention  we  can  reduce  the  number 

2  3 

of  necessary  operations  by  a  factor  of  10  or  -10  ,  provided  we  are 
willing  to  risk  a  decrease  in  the  quality  of  system  performance  in 
order  to  obtain  a  solution  to  the  computational  problem.  While  we 
realize  that  more  efficient  search  procedures  exist  than  the  complete 
examination  of  the  entire  grid  it  should  be  realized  that  we  are 
allowing  only  10  sec  for  the  location  of  the  minimum  of  a  function 
in  I4  space  in  the  time  comparisons  above.  In  addition,  in  the 
-case  of  1  or  2  irmnipula ted  variables  idiere  the  objective  fiinction 
in  concave,  Fibonacci^^^  search  techniques  can  be  used  which  are 
extremely  efficient. 
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6.1  Model  1 

We  begin  by  considering  s  problem  where,  as  before,  the 
over-all  system  dynamics  are  given  byr 

^t  +  1  =  A  yt  ■*'  ®t  +  1  ^t  -f  1  (6.1) 

y^  =  "c  Trtiere  the  have  the  same  properties  as  outlined  in 

Chanter 


In  this  section  we  will  not  concern  ourselves  with  the 
utility  functional  of  the  second  level  system  but  instead  we  will 
concentrate  on  examining  the  relationshin  between  the  first  and 
second  levels. 


System  1  is  given  the  following  problemi 
«  min  E 


-l^-l 


®1> 


t  =  1 


(6.2) 


where  ^  ^  1 


-1 

Yo  =0 


t  —  0  j  %  •  •N  “*  1 


The  vector  sequence  can  be  viewed  as  a  sequence  of  aprice* 

or  '’accounting"  vectors  i«rtiich  are  set  by  the  second  level  and  which 
snecify  to  the  first  level  unit  the  "cost"  of  a  unit  of  each  of  the 
comoonents  of  the  control  vector  m^*  An  exoression  similar  to 

(6.2)  holds  for  System  2.  The  dimensionality  of  9^  is  assumed 

—1  2  ^ 
to  be  the  sime  as  m^^  j  similarly  for  and  m^  . 
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The  over-all  operetion  of  the  system  described  here  proceeds 
as  follows t 

1«  Information 

a)  System  3  (Second  Level)  knows  E  (  • 

b)  System  3  knows  A. 

c)  System  3  knows  complete  structwe  of  first  level 
systems,  i.e.  (6.2)  and  its  counterpart  for  System 

2  *  _1  _2 

d)  First  level  units  know  and  respectively. 

e)  First  level  units  know  (6.2)  and  counterpart 
respectively. 

2.  Operation  ^ 

a)  First  level  units  optimize  with  srbitrary  . 

b)  First  level  units  communicate  to  System  3 

”t*  , 

c)  Second  level  unit  sets  Hr  so  as  to  optimize 
over-all  system  performance. 


Now  we  will  actually  solve  the  problem  specified  by  2a) 
because  of  the  rather  interesting  nature  of  the  solution. 


Due  to  the  fact  that  the  structure  of  the  solution  is 
the  same  for  both  first  level  units  we  will  drop  the  complicated 
sub  and  superscrioting  used  in  (6.2). 


We  let  Sj^  (^)  be  the  minimum  expected  value  of  the  per¬ 
formance  functional  given  that  we  have  k  decisions  yet  to  make 
and  given  that  we  follow  an  optimal  policy  from  the  current  state  "c. 


Sl.  (c)  =  min 

mi 


j(k  c  +  m^  +  B  (A  c  +  m^  +  C^) 


(6.3) 


+  2H, 


T  -  . 

m^  -f 


1 


(Ac  +  m^ 


- 
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Similarly 


(c)  =  ndn  E  |(A,  c  ♦  +  S'^f  B  (A  o  C6«U) 


”n 

-1 


=  -  B  -  (A  c  +  p.j^) 


(6.5) 


Extending  this  process  recursively  (See  Appendix  (B)  )  we 
obtain  the  following  expression  for  the  sequence  of  optimal  control 
vector: 

^  _  Ic  ^  ®  (A*  ^  1  -  3^  _  jj)  “  (A  c  _  jc)  (6.6) 

Ic  —  1| .  « «N  “  1 


as 


This  has  some  interesting  features.  First,  since  there  are 
many  components  in  as  in  implies  that  this 


method  of  intervention  does  not  reduce  the  controllability  of  the 
original  system.  Therefore,  if  the  system  manager  could  have  achieved 
a  given  performance  with  the  integrated  approach,  then  the  second 
level  unit  can  achieve  the  same  performance  with  this  mode  of  inter¬ 
vention  . 


and  3^  are  set  by 

the  second  level  \init,  they  must  be  communicated  to  the  first  level 
unit  prior  to  the  time  it  implements  However,  in  order  that 

the  second  level  unit  be  able  to  sot  3^  lc+1’  customarily  reqxiires 
knowledge  of  the  state  of  the  system  after  stage  N-k.  Therefore 


Second,  since  the  values  of  ^ 
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if  the  second  level  unit  does  not  predict  the  optimal  value  of 

and  communicate  it  to  the  first  level  system,  then  the 
first  level  system  is  forced  to  do  this  prediction. 

If  we  interpret  this  result  in  an  organization -like  context 
we  see  (at  least  for  this  class  of  dynamic  problems)  that  the  planning 
fxmction  for  the  manager,  i.e.  prediction,  is  a  natural  outgrowth 
of  the  organizational  structure  and  the  environment. 

6.2  Model  2 

In  this  section  we  consider  a  sli^tly  different  form  of 
performance  functional  for  the  first  level  systems.  Ife  will  see  that 
the  qualitative  performance  of  the  organizational  structure  changes 
abruptly  from  that  of  Model  1  contrary  to  what  we  mi^t  expect. 


\  -  k  +  1 
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System  1 


Ej* 


=^■1"  J. 


t  =  1 


'11  “t 


mi) 


(6.7) 


are  scalars  set  by  the  second  level. 

1^  +  1  (6.2). 

Syst«n  2 

gg*  =_2‘mln_2 

™1» « •  .“jj 

-2  _  .  -2^-2'  .  -2 
+  1  ^22  2^t  ■*■  “t  +  1  ■*’  fit  +  1 

Again  the  optimal  control  vectors  for  the  first  level  units 
will  have  the  same  structure  and  hence  we  drop  the  unnecessary  sub¬ 
scripts  before  we  begin  to  derive  a  solution. 


E  C  ^2^^  h2^t*  ^22 

t  =~i 


As  in  Model  Ij 


,  (6.9) 

Sjj  (c)  =  mp  E  j  y^  B  y^  +  ^  C  1;;^  ♦  8jj_j  (A  c  +  +  €^)^ 


and  (c)  =  min  E  fyj  B  y^  ^  mj  C  m^,  ] 


.-1 


®  (A  c+il^) 

yjj  =  [l  -  (B  +  C)"^  b]  AC-  [b  +  ^  ^  B  iTjj  +  f ^ 

•n 


(6.10) 


1 


6U 


Hence  by  substitution  -we  see  that  (o')  is  of  the  f<»™i 

(6.11) 

where  is  a  matrix,  b^  is  a  vector  and  d^j  is  a  scalar. 


Hence 


Sg  (c)  =  rain  E  /(A  c  +  mjj  ^  +  gjj  _  ^)T.  B  (6.12) 
\-l 


°  +  "n  -  1  ♦  e"N  -  i)  A  -  1  \  -  1  \  -  1 

+  (Ac  +  m^_^4g^_.|^)  A|j(Ac  +  i|^_^+gjj_^) 


*  ^  .  1  *  Sn  .  l)  *  \ 


Taking  the  expectation  and  differentiating,  we  have 

'^.1  =  (B+  A^)"^  [b  (A  c  4.  jTjj  _  (6.13) 

♦  A^  (A  c  ^  _  l)  4.  ] 

7 

0 

If  we  examine  (6.9)  »nd  (6.13)  carefully  we  can  detect 
some  interesting  aspects  of  the  structure  of  these  solutions.  We 
see  that  enters  into  55*  and  3J^  ^  contains  ^  . 

However  mJJ  ^  also  contains  in  the  terras  A^  and  . 

Similarly  we  see  that  m*  ^  will  contain  P ^  and  2< 


In  other  words,  the  second  level  system  must  predict  the 
"price"  vectors  from  the  current  point  in  time  to  the  end  of  the 
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decision  horizon.  This  mi^t  look  at  first  glance  as  thou^  it  would 
overldad  the  second  vmit  to  the  point  where  it  would  make  the  model 
entirely  useless.  Actually,  however,  it  opens  up  possibilities  for 
a  large  number  of  interesting  alternatives.  One  of  these  is  an 
adaptive  prediction  scheme  where  the  second  level  system  gives  initial 
predictions  which  allows  the  first  level  units  to  find  the  first  few 
control  vectors  and  as  the  process  proceeds  the  second  level  unit 
would  update  its  predictions  based  on  further  information. 

Analytically  this  model  is  quite  cumbersorae  and  hence  we 
will  not  nursue  the  maninulation  further. 


We  will  now  explore  a  class  of  models  which  reduce  the 
search  problem  associated  with  optimization.  Model  1  does,  however, 
show  us  that  we  can  achieve  the  true  optimum  in  performance  with 
indirect  intervention  if  we  are  willing  to  use  enough  components 
in  the  c[^  vectors. 

6.3  Model  3 

We  now  consider  a  model  where  the  second  level  sets  the 
internal  prices  cT^  only  once  during  the  N  stage  process. 


System  1 


_  (  ^  -1'^  _  -1  .  -,1  -1 

®  j  IZ  l^t  ®11  l^t  + 

(t=l 


(6.1];) 


where  1  = 


-1  ^  J. 

y.- 


t  +  1 


€t  +  1 
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Similarly  for  Systam  2 

*f  C  ft  ®22 

S?  (t»l  -J 

2^t  +  l“*22i^*'^*l*^*l  5?“^ 

Tor  System  3t 

3^t  ®11  3^t  ■*' 3^t  ®22  3^^  (6.16) 

'i 

3^t  +  1  ^11  3^t  ■*■  ^12  -3^1  “t  +  1  ■*■  ^  >  1 

—1  — ^  —2 

3^t  +  1  ”  -^21  3^t  ■*■  ^22  3^t  ■*■  ®t  +  1  ■*■  €t  ♦  1 

From  (6.6)  upon  deleting  the  subscripts  we  have 

(*u-u^-*iiyi-i-fi  (^-u) 

This  can  now  be  substituted  into  (6.16)  and  the  optimization  carried 
out. 


It  is  easily 'seen  that  the  model  Just  considered  signifi¬ 
cantly  reduces  the  computational  problems  for  the  system.  First 
of  all,  each  of  the  first  level  systems  is  working  with  a  model 
with  approximately  one-half  of  the  original  variables.  The  second 
level  unit  is  required  to  carry  out  only  one  outimization  in  the  N 
periods,  instead  of  N  as  in  Model  1. 

It  may  be  of  interest  for  the  purposes  of  exposition  to 
interpret  Models  1  and  2  in  an  organizational  context.  In  Model  1 
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o»ch  worker  (first  level  unit)  is  paid  a  fluctuating  salary  which  may 
be  different  from  day  to  day.  The  manager  manipulates  his  salary  in 
order  to  get  tlie  optimal  amount  of  work  from  him.  In  the  second 
model  each  worker's  salary  is  set  once  and  for  all  (for  the  N  periods), 
but  again  the  manager  would  like  to  set  the  salary  structure  in  a 
way  idiich  will  yield  the  best  system  performance.  This  second  model 
might  well  form  the  basis  for  a  model  of  "management  by  exception", 
where  the  manager  reviews  the  salary  structure  from  time  to  time  and 
adjusts  the  salary  structure  based  on  a  review  of  the  over-all  system 
performance. 

Summar;r  and  Conclusions 

In  this  chapter  we  examined  three  different  models  with 
an  eye  to  reducing  the  dimensionality  of  search  in  the  optimization. 


Model  1.  In  this  model  we  found  that  if  we  were  willing 
to  include  enough  components  in  the  "intervention  vector"  that 
we  could  achieve  the  same  result  with  decentralized  control  as  with 
the  integrated  approach.  We  also  found  that  for  this  class. of 
models  the  planning  function  arose  naturall^y  as  a  result  of  the  ory 
ganizational  structure. 

Model  2,  Here  we  investigated  a  model  which  was  a  sli^tly 
modified  version  of  Model  1.  Here  it  was  found  that  the  qualitative 
characteristics  of  the  system  operation  changed  abruptly.  The 
second  level  unit  was  required  to  nredipt  the  "prices"  for  the  first 
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level  systems  over  the  entire  decision  horizon  instead  of  just  one 
period  ahead  as  in  Model  1. 

Model  3«  Here  the  second  level  set  one  epricett  vector  for 
each  of  the  first  level  units  and  then  left  them  alone.  This  pro¬ 
cedure  significantly  reduces  the  dimensionality  of  the  search  problem. 

NO  models  were  formulated  for  the  derivation  of  the  optimal 
internal  "prices"  by  the  second  level  units.  This  is  an  entirely 
straight-forward  process  and  would  proceed  in  a  manner  similar  to 
the  formulation  of  the  models  for  the  first  level  units.  It  should 
be  pointed  out  again  that  the  second  level  unit  will  have  to  include 
all  of  the  original  state  variables  in  its  optimization,  but  that  the 
search  will  be  over  a  set  of  significantly  reduced  dimensionality 
(in  Models  2  and  3)* 
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7.  INDIRECT  MODEL  -  DIRECT  INTERVENTION 


Introduction 

The  models  considered  oreviously  have  had  a  common  pro- 
pertyt  They  all  required  that  the  second  level  unit  have  complete 
knowledge  of  the  stnicture  of  all  the  systems  below  it  and  in 
addition  that  the  first  level  units  communicate  their  decisions  to 
the  second  level  before  they  were  imolensented.  Clearly  this  places 
a  great  "information  load"  on  the  managerial  unit.  It  is  the  pur¬ 
pose  of  this  chapter  to  place  a  correspondence  between  solving  a 
particular  class  of  non-linear  optimization  problems  and  the  operation 
of  a  decentralized  organization.  The  algorithm  developed  for  the 
optimization  problem  will  make  possible  the  synthesis  of  an  organi¬ 
zational  structure  which  significantly  reduces  the  problems  of 
"information  overload". 

7.1  The  Programming  Problem 

We  will  consider  problems  of  the  fcllowi,ng  type. 

subject  to  (7.1) 


K  >  0 


—1  —2 

max  g,  (m  )  ♦  g  (m  ) 

J.  -2 
i  ,m 

Ai  (^)  5 
« 

h  ♦  1  ♦  1  5" 

_2 

®i  (?)  5^1 

« 

3^  (i^)  5b^ 


bi  >  0 
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where  ...nijJ  ,  m  =  ^  I’***™]!!! 

gjl^,  -A^»  -B^  »re  all  assumed  to  bepfositlve  concave  functions’ of  their 
respective  arguments.  In  this  section  we  will  develop  a  NDecomposi- 
tion«  principle  for  the  solution  of  (7,1)  so  that  the  problem  can  be 
broken  into  subproblems  and  the  resultant  solution  of  these  sub¬ 
problems  will  then  be  "coordinated**  to  yield  a  solution  to  ( 7.1). 


The  subproblems  arei 


max  (m^)  +  X  ^  (S^  -  Aj  ^  (m^)) 


m 


,^1. 


subject  to  (m  )  <*2 

A^  (m^)  <  a^ 


max 

■_2 


2:  -2 
Aj  (m  )  <  a^ 

,g2  (m2)  .K  (82,-B^  ^  ^  (^n 


C7.2) 


(7.3) 


subject  to  (m  )  <b^ 

4 

(S=)  <b^ 


The 


^  2 

terms  (Sj^  "  ^j  +  1  (“  ))  ^2  (Sg  -  ^  ^  (m  )  ) 


require  some  explanation.  The  factors  *^nd  Lagrange 

multipliers,  and  Sg  are  variables  such  that  0  <■  <  K> 

0  <82  <  K  ,  By  a  "formal  solution"  to  problems  (7*2)  and  (7.3) 
we  mean  m^*  A*  (S^)  and  S^*  (Sg)*  A*  (Sg)  respectively, 


f 
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where  ra  (S^)  is  the  vector  of  optimizing  values  of  m  as  a 
function  of  the  value  of  and  A*  (Sj_)  is  the  value  of  the 
lagrange  multiplier  at  the  optimal  point  as  a  function  of  • 
Similar  remarks  hold  for  m  (82)  and  A2  (Sg)  • 


We  now  require  the  following  lemma  on  Lagrange  multipliers. 
Lemma  7"1«  Given  the  problem 

max  g  (E^m^)  +  A  (S,  -  A  (E^)  )  +  A,  (Sj  -  B  (E^)  ) 

-1-2  ^ 
m  |in 

irtiere,  as  above,  0  <  S^^  <  K  and  further  g,  -A  and  -B  are 
concave  continuously  differentiable  functions  defined  over  the  posi¬ 
tive  orthant  of  (Euclidean  n  space). 

Define  g*  (S^^.Sg)  5  g  (E^*  (S^),  m^*  (S2)  ) 


df*  =  K 

z: 


98 


1-  j  =  1 


ag 


?m. 


V 


^  -  A* 


d  m. 


ds 


m. 


=  53  ag  ^  a:  - 
J  =1^ 


3m*  dSi 


=  X 


*  "  E 


(jg  _ 


j  =  1  I  dm* 


A*  dl_ 

dm;l  ds. 


(7.U) 


(7.5) 
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Then  whenever  _ si_  is  finite,  we  have - =  A,  because 

dS^  3S3_ 


..  dA 

3  m* 

vl 


1  ♦ 

vi 


=  0  is  a  necessary  condition  for  the  existence  of 


ag* 


«/ => 

an  extremum  (See,  for  example,  Karlin''  Similarly  - -  ~  A2 


ds 


2 


O  \  #  M. 

whenever  - d.  is  finite.  Further  >  0,  i.e.  g*,  is  a 


as 


monotone 


function  of  . 


Now  referring  to  (7«1)  w«  would  like  to  replace 
g  =  (m^)  -f  gg  (m^)  with  0  =  (S^)  +  Og  (Sg)  where 

Oi  (Sj^)  =  (m^*  (S^)  )  . 

Hence  we  require 


Theorem  7-1 


thenr 


^1  ^Si)  =/^Ai  dT  ♦  gi 


Oj,  (Sj)  =  2  (T)  dr  +  gg 


C7.6) 

C7.7) 


and 
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proof*  From  Lemma  7-1  we  know  that 


ii  =  (3i> 


and 


^0 


=  ;^2 


(7.8) 


(7.9) 


and  ftirther  we  know  by  inspection  of  (7.1)  that  0  =  0^  +  O2  . 

integrating  (7.8)  we  find  Q  =  J"  Ai  i^)  ^  (^2)  where 

o 

H  (Sg.)  is  an  arbitrary  function  of  .  But  (7*9)  tells  us  that 

H  (S2)  =  J’  ^2'  *  C  where  C  is  an  arbitrary  constant, 

o 

But  the  hypothesis  of  the  theorem  allows  us  to  evaluate  C  as 

gl  (S)  ♦  gg  (^)- 

Problem  (7.1)  can  now  be  reformulated  as 


max 

®1’®2 


0l  (S^)  +  O2  (Sg) 


subject  to  0  <  ♦  Sg  •<  K|  >  0 


(7.10) 


There  are  several  sifficulties  which  may  have  already  come 
to  the  reader’s  attention.  The  first  question  which  arises  is, 

"In  what  sense  is  (7. 10)  equivalent  to  (7.1)?"  We  might  know  the 
optimal  values  of  the  and  not  know  m  ,  This  is  due  to  the 

fact  that  we  eliminated  quite  a  lot  of  information  in  arriving  at 
(7.10).  Secondly,  there  is  the  problem  of  non-uniqueness.  W®  wrote 

V 

m  (S^)  as  though  it  were  a  specific  value,  but  actually  it  is  a 
set  of  values  for  there  may  be  many  vectors  m^  which  optimize  (7.1) 


7k 

for  any  specific  allocation  of  the  s^.  Third,  the  analytic  diffi- 
ciilties  in  arriving  at  (7«l0)  Kiill  generally  become  insuperable 
for  most  problems.  In  the  next  section  we  propose  an  operational 
procedure  interpreted  in  the  context  of  the  theory  of  the  firm 
Trtiich  overcomes  these  difficulties. 

7*2  Interpretation  and  Operation 

We  assume  that  (7*1)  arises  in  the  following  context. 

The  performance  functional  is  the  profit  function  for  a  firm  with 
two  operating  divisions,  i.e.  is  the  profit  function  for 

division  1  and  is  the  profit  function  for  division  2.  The 

constraints  1  <  i  <  j  are  given  by  the  technology  of  divi¬ 
sion  1.  1  <  i  <j2.  are  similarly  defined  for  division  2.  The 

constraints  Aj  ^  ^  ^  ^  related  to  the  operation  of  the 

firm  as  a  whole.  For  example,  this  constraint  might  be  the  expression 
of  the  fact  that  the  total  quantity  of  ooerating  capital  available 
is  equal  to  K  .  One  aooroach  to  the  management  of  this  firm  is 
to  centralize  the  operation  and  five  one  organizational  unit  the 
task  of  solving  (7«1)  and  setting  the  production  schedules  for  the 
operating  divisions.  This  corresponds  in  mathematical  formalism 
to  a  direct  solution  of  (7.I).  A  second  approach  to  the  management 
problem  is  to  set  up  a  planning  group  for  each  operating  division 
and  a  central  planning  unit  to  coordinate  the  activities  of  these 
divisional  planning  groups. 
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Henc®  in  the  delegation  of  responsibility  yre  give  division 

1  the  task  specified  by  (7.2)  and  division  2  the  task  of  solving 

(7*3)*  The  central  planning  unit's  (unit  3)  job  is  to  allocate  the 

total  constraint  K  between  the  two  ooerating  divisions  in  such  a 

way  as  to  maximize  the  total  nrofit  for  the  firm,  i.e.  Maximize 

gl  +  g2  •  Now,  if  wo  examine  (7.IO)  carefully  we  see  that  this  is 

the  mathematical  formulation  of  the  central  planning  unit's  problem. 

Hence  the  management  of  the  firm's  operations  proceeds  as  follows j 

Step  1.  The  two  first  level  units  solve  (7.2) 
and  (7.3)  for  arbitrary  0  <  S .  <  K  and  the 
Lagrange  multinliers  are  coimnuni'cated  to  the 
central  planning  unit. 

Step  2.  The  central  planning  unit  then  uses  the 
Lagrange  multipliers  to  construct  and  solve 
(7.10).  The  optimal  S's  are  communicated  to 
the  first  level  unit. 

Step  3«  The  solutions  to  (7*2)  and  (7.3)  cor¬ 
responding  to  the  optimal  S's  are  implemented. 


This  procedure,  by  construction,  has  the  oroperty  that  it 
yields  the  solution  to  (7.1)«  Hence  in  the  spirit  of  the  previous 
chapters  we  would  say  that  the  performance  of  the  dencentralized 
system  is  as  good  as  the  integrated  (centralized)  system. 

Perhaps  at  this  point  a  brief  specific  example  will  aid 
the  reader's  intuition.  Corresponding  to  (7.1),  consider 
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MX  g  =  Mg  4.  (  7.1) 

?  li 

subject  to  +  Big.  +  <  10 

in^  <  2 

nil  >  0 

W*  will  aaptrate  the  problam  betwean  nig  and  and  hanca  wa  sea 

that  the  problem  satiafiaa  all  of  the  necessary  conditions  for 
separability  (see  7o). 


max 


c  gl  =  ”>1  ®2  ^1  -  ”2) 


(7.2)^ 


m^  >  0 


i  =  1,  2 


2  li 

MX  gg  =  m  +  Ag  (Sg  - 


(7.3)^ 


subject  to  m^  +  <  2  i.e.  m^  <  1 


m^  >  0 


From  (7.2)^  and  (7.3)^  ^1 


Sg  <  1 


(7.10)'*'  becomes 

m*x  g-  =  .  2  Sj^ 

i\fr 


3/='  *  s.  3/2 


subject  to  0  <  +  Sg  <  10 


Sg  5  1 


Si  >  0 


(The  objective  function  g  is  not  concave,  however  in  this 
case  we  will  be  able  to  handle  the  entire  problem  analytically  and 
we  will  not  need  to  employ  the  Gradient  Method. ) 
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The  solution  to  (7.10)^  is  S*  =  10,  sj  =  0  which  yields 
m*  =  0  ,  m*  »  “2  "  ~  “i  substituted 

1  A 

into  (7il)  ia,  in  f»ot,  equal  to  g*  , 

This  procedure  will  become  increasingly  complicated  as  the 
complexity  of  (7«1)  increases  and  hence  we  require  a  numerical 
procedure,.  We  assume  that  problem  (7.2)  and  (7.3)  can  be  solved 
numerically  via  the  use  of  standard  linear  or  non-linear  programming 
techniques. 

We  address  ourselves  here  to  the  problem  of  the  second 

level  unit.  This  unit  receives  the  numerical  values  of  the  respective 

Lagrange  multipliers  and  from  this  information  alone  it  must  decide 

on  a  method  for  changing  the  values  of  the  relative  allocations  of 

the  'S^  .  For  the  case  where  and  are  corcave  functions  of 

their  respective  arguments  the  "Oradient  Method"  furnishes  us  with 

just  such  a  procedure.  For  a  complete  description  of  this  method 

and  its  stability  and  convergence  properties  we  refer  the  reader  to 
(2) 

the  literature.'  Operationally  the  \xse  of  the"  "Gradient  Method" 
would  proceed  as  follows: 

Step  1.  The  central  planning  unit  specifies 
initial  allocations  of  and  Sg  .  We  will 

denote  these  by  and  and  =  K 

for  i  =  1,  2  ... 
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Step  2,  The  first  level  un: 
(7«3)  with  S,replaced  by  S 

s?>. 


1^. 


solve  (7*2)  and 
Sg  replaced  by 


Step  3«  The  ntimerical  values  of  the  Lagrange 
multipliers  and  are  transmitted  to 

the  central  planning  unit. 


Step  U .  If  AW  then  the  value  of 

=  sW  .  sW  = 


This  process  proceeds  until 


at  Tdiich  time  the  optimum 


has  been  reached  at  an  interior  point  or  it  proceeds  until 

Ai  =  K,  =  0  «nd  or  B>  =  0,  =  It 

.„d  the  extremxim  occurs  at  an  end  point.  To  see 

more  clearly  where  the  term  -  ^2  comes  from  in  Step  U  we  notice 

that  0  (Sj^jSg)  =  (S^)  +  O2  ($2)  =  {5{)  *  O2  (K-S^) 

^s^  3s^  ^82  ds^  95^  ds^  ^  ^ 

Therefore  the  changes  in  the  Sj^  are  made  proportional  to  the  deriva¬ 
tive  of  the  total  profit  function,  hence  the  name  ^Gradient  Method." 


We  note  in  passing  that  we  can  accommodate  some  relaxation 
of  the  conditions  posed  in  the  formulation  of  (7»1)«  First  we  can 
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acconunodate  more  than  one  coupling  restriction.  Just  how  many  more 
is  not  an  easy  question  to  answer,  since  it  depends  on  the  computa¬ 
tional  comnlexity  of  the  specific  problem.  Second  we  could  with 
ease  extend  this  to  more  than  two  divisions.  Third  we  could  accom¬ 
modate  any  utility  function  on  the  cart  of  the  central  planning 

—1  —2 

unit  of  the  form  H  ^  resultant  computa¬ 

tional  problem  of  maximizing  H  is  feasible.  Note  that  H  need 
not  be  concave  or  convex.  This  again  brings  up  the  interesting 
possibility  that  the  central  planning  unit  may  give  the  separate 
divisions  erroneous  profit  equations  to  work  with  to  expedite  the 
operation  of  the  entire  firm.  The  actual  feasibility  of  this  pro¬ 
cedure  has  to  be  evaluated  in  the  particular  context  of  the  specific 
problem.  Fourth  we  can  at  least  open  the  door  to  and  formulate  the 
problem  of  more  general  structures  than  have  previously  been  con¬ 
sidered  in  this  paper,  i.e.  the  problem  of  multiple -level  structures, 
e.g.  m<  3«  In  our  problem  K  was  considered  to  bo  fixed  by 
harsh  reality.  In  fact,  however,  this , could  have  been  set  by  a 
hi^er  level  than  the  central  planning  unit.  The  hi^er  level  unit 
then  has  the  responsibility  for  coordinating  2  or  more  (3-unit) 
systems  of  the  type  wo  consider' hero.  In  the  next  section  we  will 
formulate  this  problem  in  detail. 


1 
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7t3  Multiple-Level  Structurea 

Consider  the  following  problemt 


max  gT'(5^)  +  gg  ♦  g^  (m^)  +  gj,  (^)  (7*11) 

-L-2-3-14  J-  <:  j  4 

ni^n^ni^n 

subject  to 

Ao<4.  1  (^)  ♦  ^  1  (5^)  +0^+1  (“^)  +  1  5  ? 

(5^)  < 


Afl(  (m  )  < 


\ni  ) 

®/S  (^)  5  V 


Cl  (m^)  <Ni 

c=^)  5^^ 


Di  (m^)  5  ?! 

K*  (^)  5 


We  will  consider  the  correspondence  between  the  solution  of  (7«11) 
and  the  problems  of  decentralized  control  which  might  arise  in  an 
organizational  structxire  of  the  following  topology. 
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0^  represents  the  3d  level  goal-seeking  unit  or  system  manager. 

and  Q22  *-'«  the  second  level  units  (middle  management).  0^^ 
are  the  first  level  \inits.  In  the  language  of  the  General  Systems 
Theory,  we  have  a  3-^’  -  7^  system. 


We  entrust  0^  with  the  task  of  solving  (7. 11),  but  we  only 
allow  it  two  manipulated  variAles.  Its  problem  is  to  split  up  the 
constraint  involving  the  quantity  If  and  to  allocate  the  total 
quantity  ?!  to  ■nd  Q22  in  a  way  that  solves  (7»ll)*  Q2j^ 

is  entrusted  with  the  Job  of  seeing  that  g^  +  gg  is  maximized  sub¬ 
ject  to  the  arao\int  of  the  constraint  If  it  is  allocated  and  subject 
to  the  restrictions  A^,  «  G22  is  entrusted  with  a  similar  Job 

with  respect  to  •  G-j-^  has  the  problem  of  maximizing  g^^ 

subject  to  the  quantity  of  the  constraint  ^  allocated  to  it  by 
its  manager  G2-j^  and  subject  to  the  restrictions  i=2,3,U 

are  similarly  constructed. 


We  will  now  describe  in  a  rough  fashion  the  way  this 
structure  might  operate,  and  we  will  assume  that  the  reader  can, 
from  the  treatment  of  7»3»  visualize  the  corresponding  mathematical 
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development.  We  notice  that  the  subsystema 

{^22*  ®13*  identical  to  the  type  of  system  treated  in  7»2. 

Step  1.  allocates  If  to  the  second  level 

units  and  Ogg* 

Step  2.  using  the  amount  of  If  allocated 

to  them  by  pursue  the  course  of  action 

outlined  in  7*2  until  the  amount  of  If  initially 
allocated  to  0^^  has  been  optimally  apportioned 

to  the  first  level  systems. 

Step  3.  The  optimal  Lagrange  multipliers  from 
the  first  level  systems  are  commimicated  to  the 
second  level  units.  The  second  level  unit 

then  sends  71*  *  ^2  third  level  unit. 

O22  sends  X*  +  ?.*  to  the  third  level  unit.  The 

third  level  unit,  in  the  same  fashion  as  the  ( 

second  level  unit  in  7.2,  using  -f 

and  j^2  ~  changes  his  allocation  of 

If  between  the  two  second  level  units. 

This  entire  process  repeats  until  the  over-all  optimum  is  reached. 
Summary 

In  this  chapter  a  "Decomposition  Principle'*  for  a  class  of 
non-linear  programming  problems  was  developed  and  numerical  methods 
were  proposed  which  converge  to  the  optimum  solution.  The  "Deconposi- 
tion  Principle"  was  interpreted  as  a  control  device  for  a  class  of 
decentralized  organizational  structures. 
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The  extension  of  the  basic  model  in  several  directions  was 
mentioned  and  a  description  of  its  application  to  a  multi-level 
structure  (m  =  3)  was  given. 

The  exact  requirements  for  the  application  of  the  numerical 
techniques  of  this  chapter  are  that  if  we  wish  to  partition  the 
optimization  variables  m^^, ...m^  into  two  sets  iti^, and 
itij^  ^  ...m^  then  the  objective  functional  must  be  of  the  form 

g  =  gl  ^  !»•••%)  where  are  concave, 

differentiable  functions.  The  constraint  functions  F  (m)  must  be 
of  the  form  A  (m^, +  B 

are  vectors  of  concave,  differentiable  functions. 


(m^  +  1»  *  *  •  )  ^  where  A  and  B 


8.  INDIRECT  MODEL  -  INDIRECT  INTERVENTION 


AN  EXAMPIE 

Introduction 

It  was  decided  during  the  course  of  this  research  to  devote 
one  period  exclusively  to  an  example  iriiich  would  model  a  hypothetical 
real  situation.  It  is  the  purpose  of  this  chapter  to  illustrate 
the  application  of  some  of  the  theoretical  concepts  introduced  pre¬ 
viously  and  to  provide  the  reader  with  an  actual  numerical  example 
illustrating  the  utility  of  these  ideas. 

The  situation  we  will  consider  is  the  followingi  consider 
the  operation  of  three  separate  water  storage  facilities  (dams)  all 
located  on  the  same  river.  These  dams  are  assumed  to  be  located  in 
an  arid  country  such  as  Spain  and  the  function  of  the  system  is  to 
store  and  allocate  water  for  irrigation,  industrial  use,  etc.  The 
hydroelectric  aspects  of  dam  ooeration  are  not  considered  here.  The 
control  problem  here  is  to  allocate  the  water  available  to  the  system 
to  the  system's  customers  over  the  dry  period  of  the  year  (assumed 
in  this  example  to  be  six  months)  in  such  a  way  as  to  minimize  some 
measure  of  total  cost.  Each  dam  is  assumed  to  have  inputs  from  stream 
flows.  It  is! assumed  that  these  inputs  can  be  described  as  a 
Markov  process  (since  we  are  working  in  discrete  time — a  Markov 
chain).  This  assunrotion  was  substantiated  by  J.  D*  C.  Little^^^^ 
who  examined  data  from  the  inputs  to  Grand  Coulee  Dam  and  found 
the  stream  flows  to  be  autocorrelated  and  essentially  Markovian. 
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Apparently  seldom,  if  ever,  in  practice  does  the  case  of  (time- 
wise)  independent  stream  flor/s  occur.  Since  we  have  3  dams  we  will 
be  dealing  id-th  3  Markov  chains  probabil¬ 

ity  that  if  the  stream  flow  was  i  units  of  water  this  period  that 
j  units  will  be  forthcoming  next  period.  The  reader  will  notice 
that  for  the  purpose  of  simplicity  and  due  to  the  fact  that  we  sub¬ 
sequently  will  need  to  employ  a  digital  computer  that  we  will  assume 
the  water  is  measured  in  discrete  units  rather  than  as  a  continuous 
variable. 


For  the  piirposes  of  visualization  it  may  help  the  reader 
to  compare  this  model  with  the  Hungry  Horse-Grand  Coulee-Bonneville 
Dam  system  on  the  Columbia  River.  Although  the  Northwest  is  hardly 
•n  arid  area  and  hydroelectric  power  is  an  important  aspect  of  their 
operation,  these  dams  are  located  in  about  the  same  relative  posi¬ 
tions  as  we  visuaU.ze  for  our  problem.  We  will  denote  our  dams  by 
number,  i.e.  being  farthest  from  the  ocean,  /2  next,  etc.  We 
assume  that  if  water  is  released  from  one  of  the  "hi^er"  dams  on  the 
river  this  water  will  reach  the  wlowerw  dams  in  significantly  less 
than  one  month.  This  is  apparently  justified  for  the  Coltimbia 
system.  The  approximate  transportation  lag  between  Grand  Coulee 
and  Bonneville  is  2U  hours. 

Each  of  the  dams’  customers  (except  possibly  those  of  dam 
m-)  are  assumed  to  use  all  the  water  allocated  to  them,  i.e.  none 
is  ever  dumped  back  into  the  river. 
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Since  we  are  concerned  with  an  arid  region  it  ia  aaaumed 
that  the  capacity  of  each  of  the  dama  la  adequate,  i.e.  dam  capacity 
is  not  a  constraint. 


The  cost  functions  for  each  of  the  systems'  customers  is 
assumed  to  be  convex  about  some  desired  quantity.  Graphically  we 
may  illustrate  this  as 
Fig.  8-1 


unit  of  water 

The  cost  functions  are  assumed  to  be  independent  from 
period  to  period  and  fixed  for  each  customer. 


Each  dam  is  assumed  to  have  three  customers,  i.e.  9  in  all. 

Ife  denote  the  cost  functions  for  these  customers  by 

12  9  i 

(m  ),  gj  (m  )  ...  (m  )  where  m  is  the  amount  of  water 

allocated  to  customer  i. 

Let  the  amounts  of  water  in  the  three  dams  in  the  current 
period  be  Z^>  2^’  and  2^  respectively.  Let  m^^  be  the  quantity 
of  water  released  by  dam  3  to  dam  2,  and  m^^  the  amount  released 

from  2  to  1.  yfe  will  indicate  the  scheduling  period  by  subscripts, 

9  M 

e.g.  m^  is  the  amount  of  water  allocated  to  customer  /9  in  the 

first  scheduling  period.  Let  be  the  amount  of  input  (from 
stream  flows)  to  dam  J  in  period  K. 
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8«1  The  Mathematical  Problem 

We  can  now  formulate  our  control  or  optimization  problem 
mathematically  as  follower 


(8,1) 


where  E  denotes  the  operation  of  mathematical  expectation.  The 
variables  are  subject  to  the  following  constraints. 


0<  m^  +  m^4'm^  +  m^^  <■  2^  (8*2) 

0  <  m^^  +  m^  +  -r  2^  ♦ 

7  p  p  ^  11 

0  <  m|  +  ♦  m^  ^  *1  ”l 

0<  +  +  -  »i  “  “i  ♦  (8,3) 

etc. 


Actually,  of  course,  this  is  a  very  clumsy  way  to  phrase  this  problemi 
A  much  neater  forumulation  is  afforded  by  Dynamic  Programming,  In 
functional  equation  form  the  problem  isr 


88 


=  ^min  Kl  (®^)  (8Ji) 


*n 


No 


IB  ^  •  •  •  m  ^  i— 1 
N, 


p(l)  pP)  pC3) 


l\_j-0  (^=0  1%^^  I%|^j  l5,^ 


-  .  1  1  2-  3  10  V, 

(a  -  m  -  m  -  m-^  -  m  +  D 

a^  -  ~ 

38  -  -  m®  -  4  n^  +  1|  |f|j^, 

N  =  1»«..C«  (..4.J. ..4)  =  Oo  The  are  the  orders  of  the 

respective  Markov  chains.  The  variables  in  (8.U)  are  subject  to  the 
following  restrictions! 

^  T  A  T 

C8‘^) 

2  .  10 


1 

2 

3 

.  10 

m 

4 

m 

m-^ 

4  m 

<  ! 

ii 
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m 
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4  m 

<  i 
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9 

3 

m 

4 

m 

4- 

m 

<  z^ 

4  m' 

The  functional  f^^  (....j....)  can  be  interpreted  as  nthe  minimum 
expected  cost  for  the  system  given  that  the  initial  state  is 

123123 

(a,a,ajX|I»I)  and  there  are  N  decisions  yet  to  make 
and  further,  that  an  optimal  policy  is  followed." 


It  has  often  been  said  that  "Dynamic  Programming  (as  a 
conceptual  device)  can  formulate  many  more  problems  than  it  can  solve 
(numerically)."  The  reader  has  just  seen  such  a  case,  pii'st  of  all, 
the  fact  that  the  problem  has  six  state  variables  leads  to  immense 
computer  storage  requirements,  and  secondly  the  maximization  problei* 
is  imbedded  in  an  11  dimensional  space  iriiich  leads  to  staggering 
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run  times  for  any  conceivable  search  algorithm.  As  we  will  see 
subsequently  this  problem  for  6  periods  using  a  very  coarse  grid 
approximation  would  entail  storage  requirements  of  on  the  order  of 
10  locations  and  run  times  on  the  Burroughs  220  of  on  the  order  of 
10^  years.  Clearly  this  is  not  a  problem  where  we  can  wait  for  a 
bigger,  faster  machine.  If  we  really  want  a  nximerical  solution  to 
this  problem  we  will  have  to  be  a  little  more  ingenious. 

The  general  approach  we  will  use  here  will  be  the  same  as 
is  employed  in  the  preceedijig  chapters.  We  will  first  decouple  the 
system  and  solve  three  separate  control  problems  and  then  recouple 
the  system  into  a  2  level-  U  goal  system  where  the  second  level  con¬ 
troller  is  responsible  for  recoupling  in  such  a  manner  that  the 
interaction  between  the  first  level  systems  is  compensated. 

8,2  Synthesis  of  the  2^-  U^System 

The  formulation  of  the  problems  for  the  three  first  level 
systems  is  a  simple  procedure.  The  manager’ of  each  dam  is  responsible 
for  the  solution  of  a  Dynamic  Programming  problem  for  his  dam, 
neglecting’  the  possibility  that  he  may  receive  water  from  the  dam 
above  or  be  required  to  spill  water  to  the  dam  below.  For  example, 
dam  solves  the  following  problem, . 
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4’’  I’j  -  15  jz:  h  .3 .  (8.6) 

4-1  °  5  ^  - 


<  a 


r««)!sO  N“  lj««»6» 

0  '  » 

A  similar  axpreaaion  holds  for  the  optimization  problems  to  be  solved 
by  dams  1  and  2.  The  (z^»  I^)  are  to  be  tabulated  for  all  three 

A 

dams  (j  =  1,  2,  3)  and  all  six  periods  (K  =  1, ...6)  and  all  feasible 
values  of  z^  and  -within  the  expected  range  of  operation.  These 
solutions  are  -then  transmitted  to  the  second  level  unit.  Only  the  f 
values  are  required  and  not  the  m^*. 


Now  we  -will  consider  the  fiinctions  of  the  Second  level 

unit.  Consider  f^  (z^,  T^)  -  f^  (z^  ♦  1,  I^).  This  is,  by  the 

defini-tion  of  f^,  the  expected  marginal  value  of  water  to  dam  1 

given  that  N  decisions  have  yet  to  be  made,  z^  units  of  water  are 

now  in  the  dam,  and  the  last  period* s  input  was  units.  By  similar- 
*11  111  1 

ly  examining  (*  ♦  2,  I  ),  (®  ♦  3»  I  )>  etc.,  we  could  by 

polynomial  fit  or  similar  raetiiod  construct  a  function  which  is 
daml»s  marginal  utility  function  for  extra  water.  Assume  this 
function  has  been  constructed,  we  will  denote  it  as  (n^^)  since 
m^^  is  where  extra  water,  if  any,  will  be  forthcoming. 

Now  dam  2  is  asked  to  solve  the  following  problem* 
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find 


4  (.^  i^)  = 


h  J  6  11/ ^  ^ 

SI  ^Bi  ^ni  |iii 


(8.7) 


‘  IT  '‘f’  »lj  'S-l  ‘'i2>  ?2>} 

J?2=0  J 

2 

Note  that  in  the  right  side  of  (8.7)  we  are  using  fjj_2  net 
2^ 

hence  this  is  a  single  optimization  problem  and  not  N  prob- 
2 

lema,  because  ,  has  already  been  tabulated. 

N**J- 

Now  exactly  the  same  process  is  repeated  for  dam  2  that 

t  2  2  t  2  2 

was  carried  for  dam  1,  i.e.  (z  ,  I  ),  (*  ♦  1»  I  )  ..«  are 

used  to  construct  Qg  (m^^)  •  Dam  3  is  now  asked  to  solve  the 
problemi  find 

I^)  =  ®in  C®^)  -  ^2 

”  1  2  3  10  /  ^  ^ 
m  ,m  ,m  ,m  f  i=l 


KfO 


1  3  -  h  '3>? 

i*»l 


The  results  of  this  optimization  are  then  implemented,  i.e.  dam  3 

'  1#  2# 
gives  m  •  units  of  water  to  customer  1,  m  units  to  customer  2, 

etc.,  and  m  units  are  spilled  from  dam  3  to  dam  2. 

Dam  2  now  finds  the  optimal  values  of  m^,  m^,  m^  and  m^^ 
2^  2'  10  2 

corresponding  tef^j  (z+m,I)  and  implements  them. 


n 


7  8  9 

Dam  1  then  flnda  the  optimal  values  of  m  ,  m  ,  m 
corresponding  to  f^  (2^  +  r^)»  These  are  then  implemented. 

The  inputs  from  the  streams  are  assumed  to  occur  at  this  point, 
and  the  process  repeats. 

The  function  of  the  second  level  unit  hero  is  the  administra¬ 
tion  of  the  .recoupling  rarocess  and  specifically  the  construction  of 
the  functions  0^^  and  from  the  separate  Dynamic  Programming 
solutions. 

Although  the  recoupling  process  is  lengthy  to  describe  it 
is  quite  efficient  computationally,  as  we  will  see. 

813  Computational  Results 

The  entire  process  described  abcTve  was  programmed  for  the 
Burrou^s  220  Computer.  The  following  simplifications  were  intro¬ 
duced  to  cut  "run"  times  to  a  minimum. 

1.  (m^)  =  (m^  -  if,  gg  (m^)  =  (m^  -  2^, 

ej  (m^)  =  (Jn^  -  3)^ 

2.  g^  and  g^  have  the  same  form  as  g^^,  g^  and 
gg  have  the  same  form  as  g^,  and  g^  and  g^ 
have  the  same  form  as  g^. 

3.  All  three  dams  were  assumed  to  have 
identical  input  Markov  chains. 
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il*  A  simple  2  point  quadratic  fit  is  used  to 
obtain  the  using  f^  (z,  I)  and 
f^  (z  +  1,  I)  and  4  (z  +  2,  I). 

The  input  process  for  each  of  the  three  dams  was  simulated 
for  each  of  six  oeriods  and  these  inputs  were  used  to  simulate  the 
decision-making  process.  The  Dynamic  Programming  solution  took 
roughly  10  minutes  on  the  220  and  thereafter  the  complete  simulation 
of  the  six  decision  periods,  including  the  random  input  generation, 
averaged  about  two  minutes.  A  tot-al  of  11  complete  six-period 
simulations  were  inn,  i.e.  66  decision  periods. 

The  following  modifications  could  have  been  a  cc  ODmodat  ed 
and  still  keep  the  D.  P.  calculations  and  one  simulation  to  on  the 
order  of  one  hour  run  time. 

1.  3  separate  D.  F.  programs  instead  of  one. 

2.  Larger  Markov  chains,  up  to  possibly  8x8. 

3.  Non-stationarj'-  cost  functions  and/or 
Non-stat ionary  Markov  chains. 

U.  More  iophisticated  fitting  procedures  to 
obtain  the 

More  peiriods  than  six. 

For  example,  the  P^^^  element  tells  us  that  if  we  received 

0  units  of  water  last  month  we  will  receive  0  units  this  month  with 
probability  .3  « 


9k 


The  method  of  search  employed  In  the  program  ifas  simple  enumerative 
search  point  by  point*  A  more  sophisticated  technique  such  as 
•'Steepest  Descent"  would  make  the  comoutation  of  much  finer  grid 
siies  possible. 

we  will  now  compare  the  storage  requirefients  and  erun"  times 
of  this  simulation  to  an.  estimate  of  the  requirements  for  the  solu¬ 
tion  of  (8.U).  We  will  assume  that  the  average  grid  size  per  state 
variable  .'.was  10,  i.e.  the  solution  was  calculated  for  10  values  of 
each  of  the  state  variab  1.3a  •  We  a  ssume  that  the  nxuiber  of  scheduling 
periods  is  six.  A  little  arithmetic  will  convince  the  reader  that 
the  storage  requirements  for  the  solution  of  (8»ii)  is  on  the  order 

7 

of  10'  memory  locations,  while  for  our  routine  the  requirements 
can  be  kept  to  well  under  10^.  Therefore  if  the  cost  of  computation 
goes  up  linearly  with  the  number  of  required  storage  locations  the 
ratio  of  the  coimputation  cost  of  (8.U)  versus  the  procedure  outlined 
in  8*2  is  on  the  order  of  10^,  i.e.  the  solution  of  (8.i*)  would  be 
at  least  a  thousand  times  were  costly  than  the  appi’oach  outlined 
in  this  chapter.  Actually,  there  are  no  existing  computing  facilities 
with  ttie  size  required  by  (8.U),  and  hence  its  solution  is  unfeasible, 
regardless  of  cost. 

we  now  turn  to  a  consideration  of  the  performance  of  our 
method.  The  following  procedure  was  used  to  find  an  upper  bound 
for  the  possible  performance.  If  one  knew  at  the  beginning  of  period 
1  exactly  how  much  water  would  collectively  be  available  to  the  dams, 
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1 

I 

I 

I 

1 

I 

I 

I 

I 

I 

I 

I 

I 

I 

I 

! 

I 


and  if  all  that  irater  were  available  in  dam  3  at  the  beginning  of 
period  1,  then  it  could  be  allocated  over  the  six  periods  in  such  a 
way  that  no  better  allocation  would  be  possible.  This  condition  is, 
of  course,  unachievable,  but  it  does  yield  a  lower  bound  on  achievable 
costs.  This  cost  is  called  the  "Perfect*  cost  and  is  compared  with 
the  costs  obtained  by  our  algorithm.  A  complete  tabulation  of 
results  of  the  11  simulations  follows. 

Run  /  1 

Period  Amount  of  Inputr 


Dam  1 

Dam  2 

Dam  3 

Cost 

1 

1 

1 

1 

27 

2 

0 

1 

2 

27 

3 

0 

1 

2 

27 

li 

0 

2 

2 

27 

2 

2 

0 

29 

6 

3 

2 

2 

11 

Total 

6 

9 

9 

Ut8 

"Perfect" 

liiU 

%  dev.  from  "Perfect" 

2.9^ 

Run  #2 

Period 

Amount  of  Inputt 

Dam  1  Dam  2 

Dan  3 

Cost 

1 

1 

1 

1 

27 

2 

2 

1 

2 

27 

3 

2 

2 

3 

27 

U 

3 

2 

3 

9 

5 

0 

3 

2 

21 

6 

1 

0 

2 

lii 

Total 

9 

9 

13 

125 

"Perfect" 

123 

%  dev.  from  "Perfect" 

l.6o;< 
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Run  i(3 


P«riod 

Amount  of  Input* 

Darn  1  Dam  2 

Dam  3 

Cost 

1 

r 

1 

1 

27 

2 

2 

1 

2 

27 

3 

3 

0 

1 

2U 

h 

2 

2 

2 

21 

S 

2 

3 

1 

18 

6 

2 

2 

3 

8 

Total 

12 

9 

10 

125 

■Perfect* 

123 

%  dev.  from  eperfect* 

1.658 

Run  $x 

Period 

Amount  of  Input* 

Dam  1  Dam  2 

Dam  3 

Cost 

1 

*»  X 

1 

1 

27 

2 

0 

1 

2 

27 

3 

2 

3 

0 

32 

U 

2 

2 

0 

23 

5 

2 

0 

1 

21* 

6 

2 

0 

0 

31 

Total 

9 

7' 

h 

161* 

■Perfect" 

156 

%  dev,  from  "Perfect* 

5.1J< 

Run  #  5 

Period 

Amount  of  Input* 

Dam  1  Dam  2 

Dam  3 

Cost 

1 

1 

1 

1 

27 

2 

1 

2 

2 

27 

3 

2 

1 

2 

27 

U 

2 

1 

3 

15 

5 

2 

1 

1 

21* 

6 

2 

1 

0 

18 

Total 

10 

7 

9 

138 

■Perfect" 

138 

%  dev.  from  "Perfect* 

0.058 

Y 

1 

97 

Run  /  6 

1 

Period 

Amount  of  Inputt 

Dam  1  Dam  2 

Dam  3 

Cost 

1 

0 

0 

2 

37 

1 

2 

0 

1 

2 

27 

3 

0 

1 

2 

27 

1 

k 

0 

2 

2 

27 

1 

5 

2 

2 

0  ■ 

2)i 

a 

6 

3 

O 

6. 

2  . 

12 

g  Total 

? 

8 

10/23 

13U 

•’Perfect" 

Ui7 

H  ^dev,  from  '•Perfect** 

1  Run  /  7 

Period 

Amount  of  Input; 

Dam  1  Dam  2 

Dam  3 

Cost 

1 

1 

0 

0 

2 

37 

I 

2 

2 

0 

2 

27 

1 

3 

2 

2 

0 

29 

1 

k 

1 

0 

3 

2U 

5 

0 

1 

3 

2k 

1 

6 

0 

_1 

1 

21 

Total 

5 

h 

11/20 

162 

H  •’Perfect" 

1^6 

*  ^dev.  from  •’Perfect*’ 

3.70J< 

H  Run  /f  8 

a 

Period 

Amount  of  Inputt 

Dam  1  Dam  2 

Dam  3 

Cost 

1 

1 

0 

0 

2 

37 

2 

1 

2 

2 

27 

1 

3 

2 

1 

2 

27 

h 

2 

1 

3 

12 

1 

5 

2 

1 

1 

2U 

i. 

6 

2 _ 1 

0 

18 

~|  Total 

9 

6 

10/23 

li,l3 

i  "Perfect" 

liil 

^dev«  from  "Perfect" 

2.78^ 

1 

98 


Run^  9 

Period 

1 

2 

3 

h 

5 

6 

Total 

wperfect" 

Jtdev*  from  "Perfect" 
Run  ^  10 

Period 

1 

2 

3 

U 

5 

6 

Total 

"Perfect" 

Jidev.  from  "Perfect" 
Run  /  11 


Total 
"Perfect" 

%dev,  from  "Perfect" 


Period 

1 

2 

3 

U 

5 

6 


Amount  of  Input* 

Sam  1  Dam  2 


0 

2 

2 

3 


0 

1 

2 

2 


0 

1 


8 


Amount  of  Input* 

Dam  1  Dam  2 


0  0 

2  I 

3  0 

2  2 

2  3 

—i 

11  8 


Amount  of  Input* 

Dam  1  Dam  2 


0 

0 

2 

2 

2 

2 

8 


0 

0 

2 

2 

0 

0 


Dam  3 
2 
2 
3 
3 
2 
2 

lii/30 


Cost 

37 

27 

2U 

9 

21 

lil 

132 

126 

h.^^% 


Dam  3 
2 
2 
1 
2 
1 

_3 

11/30 


Cost 

37 

27 

21 

21 

18 

8 

132 

126 

U.55X 


Dam  3 
2 
2 
0 
0 
1 
0 

5/17 


Cost 

37 

27 

37 

26 

2U 

_31 

182 

167 

8.25X 
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For  the  first  five  runs  the  average  percent  deviation  from 
•Perfect"  is  2.25?.  For  the  last  six  runs  the  average  percent  de¬ 
viation  from  "Perfect**  is  U«73^«  The  performance  of  the  solution 
to  (8.1;)  ntust  lie  somewhere  betwetm  the  performance  of  our  algorithm 
and  "Perfect,"  and  hence  our  algorithm  performs  very  well  indeed. 

Oxir  algorithm’s  performance  is  less  than  2.2^  from  the  optimum 
achievable  performance  for  the  first  five  runs  and  less  than  ho73% 
from  the  optimum  in  the  last  six  runs. 


Summary 


In  this  chapter  we  have  presented  a  complete  numerical 
analysis  of  an  example  of  a  2 1*^/' system  idler e  the  second  level 
unit  employs  an  indirect  model  of  the  first  level  systems  below  it 
and  intervenes  in  the  affairs  of  the  first  level  systems  via 
indirect  Intervention.  The  model  employed  by  the  second  level  is 

r(^)t  instead 


indirect  because  it  requires  only  a  knowledge  of  f^ 

i 


(*»  1) 


of  a  complete  knowledge  of  the  dynamics  of  system  i.  The  inter¬ 
vention  is  indirect  because  the  operation  of  the  second  level  unit 
consists  of  inserting  the  factors  0^  and  0^  in  the  performance 
functionals  of  dam  2  and  dam  3  respectively. 


The  optimization  problem  was  solved  and  the  total  opera¬ 
tion  was  simulated  over  66  decision  periods.  A  lower  bound  on  the 
costs  is  generated  and  it  is  shown  that  the  algorithm’s  simulated 
performance  usually  falls  within  3  to  of  this  lower  bound. 
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It  is  also  shown  that  the  (computation)  cost  advantage  of  this 
algorithm  over  a  complete  Dynamic  Programming  solution  is  on  the 
order  of  a  thousand  to  one. 


I 

1 

1 

1 

1 

1 


T 

i 
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9.  CONCLUSIONS 

m  this  thesis  we  have  investigated  a  nmber  of  "organiza- 
tionalN  systems  or  •ajL  -  systems  where  in  each  case  the  over-all 
system  task  is  the  solution  of  a  large  multi -variable  optimization 
problem.  Our  principle  effort  was  directed  toward  the  development 
of  efficient  ecoordination'*  and  •modeling*  techniques  for  the  higher 
level  or  •managerial"  units. 

m  Chapter  3  we  introduced  the  concept  of  direct  interven¬ 
tion,  and  we  found  that  for  certain  classes  of  problems  the  perfor¬ 
mance  of  the  synthesized  2^  -  system  was  the  best  aichievable  and 

that  for  all  the  deterministic  models  of  Chapter  3  the  direct  inter¬ 
vention  approach  was  better  than  simply  neglecting  the  interaction 
between  the  first  level  units.  The  limitations  of  this  method  in 
the  static  case  are  that  it  requires  special  physical  conditions 
for  its  implementation,  it  places  heavy  information  processing  loads 
on  the  managerial  unit,  and  it  does  not  readily  extend  to  more  than 
two  levels. 

In  Chapter  U  we  introduced  indirect  intervention.  Again 
this  approach  yields  better  performance  than  one  obtains  by  reticu¬ 
lating  the  system  and  then  neglecting  the  interaction  between 
reticulated  parts.  This  irethod  requires  no  special  conditions  for 
its  implementation,  and  further  it  sii?nificantly  reduces  the  computa¬ 
tional  l»ad  on  the  managerial  unit.  Htwrever,  this  method  does  reduce 
the  controllability  of  the  over-all  system  as  viewed  from  the  point 
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of  view  of  the  hipest  level  unit.  This  in  itself  is  no  handicap 
because  it  is  possible  for  the  second  level  unit  to  have  very  few 
controllable  variables  and  still  be  able  to  steer  the  performance 
of  the  overnall  system  to  the  optimum. 


In  Chapter  5  we  found  that  the  direct  intervention  approach 
applied  to  a  broad  class  of  linear  dynamic  systems  leads  us  to  the 
conclusion  that  in  this  case  the  2^-  3^  system  is  equivalent  to  a 
1^-  2^  system. 


In  Chapter  6  we  employed  indirect  intervention  in  the 
dynamic  case  and  showed  that  it  can  be  used  to  significantly  reduce 
the  dimensionality  of  the  search  problem.  We  also  were  able  to  de¬ 
monstrate  that  prediction  can  arise  in  an  eorgc-mization"  purely  as 
a  function  of  the  structure  of  the  system.  The  planning  function 
in  a  multi-level  system  need  not  be  linked  to  environmental  uncer¬ 
tainty  at  all,  but  may  be  built  into  the  system  as  a  control  device. 


In  Chapter  7  we  introduced  the  concept  of  the  indirect 
model.  In  Chapters  3 j  'U»  5  and  6  the  second  level  unit  is  assumed 
to  have  complete  knowledge  of  the  entire  system  and  its  environment, 
i.e.  it  has  a  direct  model  of  the  system  below  it.  In  Chapter  7 
only  one  nxmiber  is  required  by  the  second  level  system  from  each 
lower  level  unit  in  any  one  decision  period  in  order  that  the  second 
level  unit  be  able  to  steer  the  complete  system  to  the  optimal  over¬ 
all  performance.  The  feasibility  of  extending  this  tyT)e  of  model 
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to  more  than  two  levels  was  investigated,  and  one  class  of  three 
level,  seven  goal  systems  was  discussed  in  some  detail. 

m  Chapter  8  we  applied  the  concepts  of  indirect  interven¬ 
tion  and  indirect  modeling  to  synthesize  a  2^  —  system  for  the 
control  of  a  3-dam  water  storage  system.  Numerical  results  were 
obtained  and  the  quality  of  performance  as  well  as  computational 
costs  were  discussed. 


9. 


loU 
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APPENDIX  A 

In  this  Appendix  we  will  prove  several  results  relating 
to  matrix  analysis  and  the  theory  of  operators.  It  is  assumed  that 
the  reader  is  familiar  with  matrix  algebra  and  real  analysis. 

We  will  define  the  following  notation 

m  is  an  n-component  constant  vector. 

2  is  an  n-component  random  vector  with 
expected  value  Eff=p  « 

A  is  an  n  x  n  constant  vector. 

Theorem  A-1.  E  (m  A  ?)  =  i  A  p, 

Proofr  Let  ?  =  5^  A  =  ig  nQ 

3?  A  S  =  ^1  **■  “2^2'  ***  ^^n 

E  A^)  =  +  i2  P2  ^  *  P  <3»B.D. 

•JT  —  T  • 

Theorem  A-2.  m  A  p  =  p  A  m 

-Jir  =  T  — 

As  above,  let  m  “mA  m=Am 

^  —  -T  = 

Therefore  we  can  write  the  theorem  as  m  p  =  p  m  . 

»^l‘ 

« 

=  ...  ^pj^ 


in 


p  =^m. 


1,  m2  ,  m3 


lOT 


=  ^Jl3^  Sg  +■ ... 

m  ^  ^ 

Theorem  A-3*  d  (m  A  m)  =  2  A  m 
d  m 


Q.S .D • 

if  a’’^'  =  A 


Proof  t  By  d  Tre  mean  the  vector  of  derivatives 
d  m 


"*1  *11  “l  ”l  *12  “2  •••  ""n  *nn  %  *  —  A  m)  =  2  + 

dm^ 

ai2  ♦  ag^  nsg  +  m^  +  a^^  m^  +  ...  m^  but 

m 

a^j  =  hence  d  (m  A  m)  =  2  a^^  4  2  a^g  mg  +  ...  2  a^  m^ 

d  m^^ 


d  m^ 

.r 

d 
d  m 


— T  - 

.  m  Am 


=  2  A^  •  i 
Similarly 


Trhere  A^  =  a 


'  I* 


11>  ai2' 


*  •  «  di 


_d _  (ra^  A  m)  =  2  A^  *  m 


d  ra . 


"] 


In 


d  (5^  A  m)  =  2 
d  m 


m 


=  2  A  m. 


106 


Example.  minimize  E  (5^  B  y)  where  jr  =  A  m  +  e  »nd  B  = 

T 

B  y  =  (A  m  ♦  e)  B  (A  m  +  ^  B  A  m 

+  m?*  A^  B  ?  ♦  B  A  m  -f  5*^  B  by  using  Theorem  A-2  we 

reduce  y^By=^A'^’BAi  +  2m?’A^BS'+  ^  therefore  by  A-l.i 

E  (y^  B  y)  =  n?"  A*^  B  A  iii  +  2  a'^  B  p  4.  E(T’^'B  £) 

d  E(3^B^'  =  2A'^BAm  +  2A'^Bir  since  E  B  ?)  is  » 
d  m 

constant.  If  we  equate  the  vector  derivative  to  0  we  have 

2  A*^  B  (A  m  ♦  (T)  =  0  If  k  jfl  0  and  B  ^  0  m*  =  -  a"^  (T 

idiere  m*  is  the  optimizing  value  of  the  vector  m» 

Now  we  will  take  the  relevant  portion  of  a  theorem  from 
Halmos^^*^^  which  establishes  the  fact  that  we  can  represent  any  linear 
operator  mapping  a  finite  dimensional  linear  vector  space  J  into 
itself  as  a  finite  matrix. 

Theorem  A-U.  The  correspondence  (established  by  means  of  an 
arbitrary  coordinate  system^  =  (x^, ...x^)  of  the  n-dimenslonal 
vector  space  X  )  between  all  linear  transformations  A  on  J  and 
all  matrices  o<.  .  described  by  A  x ,  =  ^ .  .  x.  is  an  isomorphism. 

Proof*  The  proof  can  be  found  in  Halmosl^'^^*®*^  We  state  without 
proof  that  it  also  preserves  the  operation  of  multiplication  (operator- 
matrix)  and  inversion. 
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!»  this  section  we  give  the  definitions  and  preliminaries 
Tthich  are  the  necessary  background  for  Theorem  li.3  of  Chapter  U. 
The  definitions  and  theorems  stated  here  are  taken  from  selected 
portions ' of  Taylor. 


Definitions. 

1.  The  space  of  all  continuous  linear  operators 
mapping  the  linear  space  into  itself  will 
be  denoted  by  X  . 

2.  If  X  and  Y  are  normed  linear  spaces,  we 
define  the  "norm"  of  the  operator  A  mapping 
X  into  Y  as 

lixll  =  sup  ||a  x||  x  je  X  where 
I  |x|  I  <  1 

I |x| I  is  the  norm  of  the  vector  x,  | |a  x| ( 
is  the  norm  of  the  vector  Ax  fc  Y« 

3«  If  ^  is  such  that  the  range  of  ^  I  -  T  is 
dense  in  X  where  T  has  both  its  donsin 
and  range  in  X  and  I  is  the  identity 
operator,  and  ±f  "21  -  7  has  a  continuous 
Inverse,  we  say  /(.is  in  the  resolvant  set 
of  T  j  this  set  of  values  of  ^  is  denoted 
by  jo  (T)  0  All  scalar  values  of^  not  in 

^  (T)  comprise  the  set  called  the  spectrum 

of  T  ;  it  is  denoted  by  (T ) « 

U*  A  space  X  is  a  Banach  space  if  it  is  a 
complete,  normed,  linear  space. 


The  Theorem  li.3  of  Chapter*U  is 
however,  somewhat  more  powerful 
state  and  prove  that  portion  of 


proved  in  Taylor. ^  It  is, 
than  we  require,  and  hence  we  will 
the  theorem  that  we  require. 


r 


uo 

Let  be  the  space  of  all  bounded  linear  operators 
napping  the  Banach  space  X  Itself* 

Theorem  A-5.  If  T  €  IA[  =►  ||t||  and  if  I  -  TT"^ 

exists  and  is  continuous,  then 

oo 

CAI  “  T)”^  y  *  y  y  for  all  y  in  the  range 

1 

of  X  I  -  T* 

proof r  If  y  =  (AT  “  I")  ^  then 

x=A  y+...  A»t  j  *■  A-  Tx 

•m  have  the  results  because  X***^  t”  x — vO  iriien  |a|  >  ||t|| 

hence  the  series  converges  geometrically  in  the  norm  of  the  space  X* 


Ill 


APPBNDIX  B 

m  this  Appendix  we  present  the  proofs  of  the  Dynamic 
Programming  theorems  used  in  Chapters  5  end  6» 


Consider  the  functional  equation 


{' 


Sk+1  («)=  “«  IS  (AC*  m^_^  ♦  B  (A  c  ♦  mjj_j^  +  (B.l) 

"v-k 


-T 


k  <=  "Vi-k  *  »'k  (*  '  *  Vk  *  %-k>} 

ahere  E  ~  ^N-k  ®e  ®  ° 


f  2.  - 

Sj  (c)  *  max  1  V(A  c  -a  mjj  +  €,,)  B  (A.  c  +  15^  ♦ 

From  (B«2)  by  taking  the  expectation  and  derivative  we  find 
=  Hjj  A  c  4  Qjj  idiere  Hjj  =  (C  -  B)”^  B 

and  Ojj  =  (C  -  B)”^  Bji^  * 


CB.2) 


ire  therefore  state  and  prove  the  following  theorems 
Theorem  B-l*  The  struct\ire  ef  the  optimal  policy  is 

n^_j^  =  Hjj  A  c  Ojj_jj  end  the  solution  to  (B«l)  is 

Sk  ^^VN-k4l  A  °  ♦  2  §  jj_,^^^  A  c  ♦  where* 
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%-k  “  ~  ^-k*  ®N-k  “  ®  *^^^  ll-k+l  * 

=  (0  -  B  'Hi  ' 

V»-k  =  (I  *  ^J'  "H-k  *  "n-k)  -  ^-k  <5  %-k 

*ll-k  ■  ♦  %-k)^  “H-k  S-k  •*  Hl-k>  -  ^-k  '>  Hwt  *  ^B-kkl 

^N4-l  ~  °  *  ~  °  “  ° 

“  ♦N-k.-l  *  2  A  CQ,_^  ^  i:^_^)  + 

®  £^*N-k  ®N-k  *  ^N-k  * 

Prooff  We  proceed  by  induction.  We  have  previously  demonstrated 
the  hypothesis  to  be  true  for  k  =  0  by  shewing 

~M  _ 

iBjj  =  Hjj  A  c  4  Ojj  and  the  hypothesis  on  (o)  is  satisfied  as  can 
be  verified  by  inspection. 

Let  the  induction  variable  be^  •  If  we  assume  that  hypothesis 
is  true  for  all  <  k-1  we  have 

\  (5)  ff’'  J  *  2 

Sk.l  ('>  '  -k  I  I  (»  ?  •*  B  (A  o  .  S;,^  .  f^.^) 

\-k  '>■ 

-  ^-k  iV-k  ‘  *  iHl-k  *  «H-k>’  *’'>l'H-k*l  *  »  '  *  »»-k  *  fH-k> 

*  *(*<=*  ♦  €b^)  * 
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T*lcing  the  expectation  and  derivative  we  find 
“  N-k  ®  “  ®N-k^  ®  ■*  *‘N-k^  ■*'  ^  ^N-k+1 


and  now  an  examination  of  *^-k  hypothesis 

on  jj  •  Substitution  of  this  result  into  C^)  verifies  the 

structure  of  Sj^  (c)  ■ 


Theorem  B-2.  The  solution  of  the  functional  equation 

^k+l  E  [CA  c  ♦  ♦  e“N_k)  E  (A  c  +  55^,^  ♦  e^_^) 

\-k 

♦  aj-k  Vk  ♦  Sj^  (A  ^  ♦  Vk  ♦  %-k)} 

where  3^  is  a  sequence  of  constant  vectors  is 

k  ~  ^N-k  ^  ^  ^-k4l  A  °  optimal  policy  is 

^_lc  “  E  (A*  -  3k-k^  -  (A  c  iiLk_j^) 

Tdiere  ♦  E  C«^N-k-fl  ■*'  %-k+l  ”  ^N-k+l^"^  ®  ^'^N-k+l 


*  ^N-k+l  "  ^^N-k+l^  * 


•k+l  "  h?-kfl 


) 


"  ^  ^-k+2  A  N-k-t.1 

•"'^‘=^N-k  =  (a'T  ^-k+1  -  Vk) 

Proof*.  The  proof  proceeds  by  induction  in  exactly  the  same  form 
as  Theorem  B-l»  and  hence  we  will  omit  it» 


APPENDIX  C 

This  Appendix  formulates  a  possible  area  of  application  for 
the  models  of  Chapters  3,  U,  5  and  6.  It  is  our  intent  to  indicate 
the  general  areas  of  application  and  to  interpret  the  features  of 
the  models  of  Chapters  3  through  6  in  the  language  of  the  application 
areaj  however,  we  will  not  concern  ourselves  with  the  specific  de¬ 
tails  of  applying  the  results. 

m  this  Appendix  we  will  interpret  the  vectors  y  and  y^ 
as  output  vectors  of  a  system  idiich  express  the  deviation  in  the 
system  outputs  from  a  fixed  operating  point,  i.e,  y-z-'a^  where 
is  the  vector  of  fixed  operating  points  and  z  is  the  actual 
system  output.  The  f  and  are  to  be  interpreted  as  vectors  of 
disturbances  which  appear  in  the  outputs  and  arise  from  causes  exter¬ 
nal  to  the  system.  The  m  and  m. ,  as  before^  are  vectors  of  manipulated 

tl' 

variables. 

An  example  of  a  system  fhere  the  dl.sturbances  appear  directly 
across  the  outputs  of  the  system  as  they  do  in  the  models  of  these 
chapters  is  in  any  electrical  generating  ^stem.  A  class  of  practi¬ 
cal  situations  irtiere  these  models  might  find  application  are  in  hydro¬ 
electric  generating  stations  where  there  are  several  turbines  operating 
off  a  single  penstock  and  where  increases  in  load  on  one  generator 
affect  the  speed  of  the  other  generators. 
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I 

I 

I 

I 

I 

I 


Tha  System  Eq^iatlona 

In  Chaptera  3  and  U  we  deal  with  system  models  of  the  type 
^  =  A  m'  ♦  C •  Here  we  assume  that  y  is  a  function  of  time  but  that 
it  depends  solely  on  m  and  «  and  not  on  its  own  past  as  do  the 
y^  of  Chapters  5  and  6.  W®  interpret  m  in  this  context  as  being 
the  magnitude  of  the  penstock  value  openings  to  the  respective 
turbines,  in  is  assumed  to  be  a  function  of  time  and  it  is  assumed 
that  the  variations  of  m  are  sufficiently  small  that  the  inherent 
non-linearities  of  the  system  do  not  influence  the  model. 

A  is  the  "system  matrix"  which  symbolically  transforms 
penstock  value  openings  into  generator  outputs.  A  is  assumed  to  be 
constant.  Mfe  will  assume  that  the  disturbances  ^  are  functions  of 
time  but  known,  i.e.  the  fluctuations  in  load  have  been  previously 
observed  and  are  periodic.  We  make  this  assumption  here  to  avoid 
having  to  deal  with  random  functions. 

The  dynamic  models  of  Chapters  5  and  6  can  arise  in  two 
different  contexts,  first  we  might  assume  that  the  system  had 
significant  internal  dynamics,  i.e,  that  the  internal  inertia  of 
the  system,  for  example,  makes  the  outputs  at  time  t+1  dependent 
on  the  outputs  at  time  t  .  Alternatively  we  might  assume  that  we 
are  dealing  with  two  different  dams  located  on  the  same  river  so  that 
the  water  available  to  the  lower  dam  is  dependent  on  the  discharge 
of  the  higher  dam. 
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The  Performance  runctlonals 

Two  of  the  most  important  goals  of  a  power  generating 

facility^^^^  are  to  mairitain  constant  oi:tput  levels  independent  of 

the  load  and  to  minimize  the  transmission  losses  which  are  a  func^ 

tion  of  the  output  levels.  We  will  assume  that  the  generating 

facility  appraises  the  cost  of  deviating  from  the  fjjced  operating, 

point  as  being  proportional  to  the  square  of  the  deviation. 

-i?  — 

We  can  state  the  first  goal  mathematically  as  minimize  y  B  y 
idiere  B  is  a  positive  definite  matrix.  It  has  been  shown  in  tlie 
literature that  the  transmission  losses  go  up  also  proportional 
to  the  square  of  the  outputs.  Hence  we  can  define  the  control 

problem  for  the  system  as 

— T  —  _  _  T  —  — 

minimize  y  B  y  +  (y  +  z  )  C  (y  +  z  )  * 
m  o  o 

(C  is  positive  definite) 


Direct  intervention 


There  are  at  least  two  ways  to  interpret  the  terms 
which  occur  in  the  output  equations  of  Chapters  3  and  One  way 
is  to  assume  that  is  a  control  signal  and  that  the  direct  inter¬ 
vention  procedure  requires  the  capability  of  taoping  into  the  generator 

sp#ed  controls  at  an  appronriate  point  in  such  a  way  as  to  introduce 
_1 

^  into  and  into  Aj^2*  ^  second  way  is  to  assume  that 

—2 

the  total  term,  e.g.  A^^g  >2,  »  inserted  directly  into  the  output. 

_2 

In  this  method  then  one  takes  the  vector  signal  ,  amplifies  it 
by  the  matrix  A^g  and  inserts  the  resultant  directly  into  the 
outputs. 


117 


Ciirrent  hydroelectric  facilities  generally  possess  the 
ability  to  inplemsnt  this  procedure  directly  due  to  the  presence 
of  lines  ehich  allow  them  to  tap  power  from  a  neighboring  power 
system.  These  lines  are  known  as  »tio  linos*. 

Indirect  Intervention 

This  mode  of  intervention  is  concerned  solely  with  the 
confutation a 1  aspects  of  control  and  hence  has  no  counterpart  in 
the  physical  ^sten. 
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APPENDIX  D 

In  this  Appendix  we  will  define  ^controllability*' in  a 
manner  irtiioh  is  suitable  for  onr  requirements  and  indicate  the  other, 
uses  of  the  tern  which  are  prevalent  ih  the  literature. 

The  concept  of  controllability,  broadly  stated,  is  based 

(12) 

on  the  followingr'  '  Given  a  system,  a  set  of  required  outputs,  and, 
a  set  of  available  manipulated  variables,  is  it  nossible  for  each 
of  the  required  outputs  to  select  a  value  for  the  manipulated 
variables  idiich  will  deliver  the  required  output? 

Brockett  and  Mesarovlc^^^  also  define  the  concepts  of 
Functional  Controllability,  Asymptotic  Controllability,  and 
Polntwlse  Controllability. 

For  our  purposes  we  will  require  only  a  sli^t  variant  of 
Polntwlse  Controllability.  For  Chapters  3  and  U  we  require  the 
following  Definition  D-l« 

The  system  modeled  by  the  state  equation  y  =  A  m  +  € 
■polntwlse  controllable*  for  every  point  7^  -  in  the 

output  space  7  it  is  possible  to  select  at  least  one  point 
in  the  space  IT  of  maninulated  variables  such  that  A  in  =  y  -  ^  . 

Several  things  become  apparent.  First,  if  the  number  of 
components  of  m  is  less  than  that  of  y  -  jp  the  system  cannot  be 
polntwlse  controllable.  If  the  number  of  components  m  is  gi*eater 
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than  or  equal  to  the  number  of  components  of  y  -  g  ,  we  must  examine 
the  matrix  A.  If  A  is  square  and  non-singular  then,  of  course, 
the  system  is  pointwise  controllable* 


lhat  occurs  geometrically  if  the  system  is  not  pointwise 
controllable?  Tot  the  answer  to  this  question  consider  the  system 
equation 


(D.l) 


where  in  this  instance  A  -  1^1  -  and  m  here  is  a  scalar 

quantity*  W«  can  regard  (D.l)  as  expressing  the  y^^  -  para¬ 
metrically  in  terms  of  the  variable  m*  Hence  (D*l)  consti'^irtes  the 
equation  of  a  straight  line  in  the  output  space  7  •  Tfhereas  if 
the  system  were  pointwise  controllable  wo  could  reach  any  point  of 
7  with  some  m  here  we  can  reach  only  those  points  on  the  straigjit 
lino  (D*l)  by  manipulating  m. 


For  Chapters  $  and  6  we  will  employ  the  following  concept* 
Definition  ’0-2*  Wo  will  define  the  system  'modeled  by  the 
equations  y^^j^  =  A  y^  ♦  5^0  ~  ®  t  =  0,  1,  *.* 

to  bo  "functionally  controllable "  < — ^overy  possible  sequence  of 
values  definable  in  the  outnut  space  7  can  be  realized  by  the 
selection  of  at  least  one  sequence  in  the  space  of  manipulated 
variables 


